Parallel Programming Techniques Research Articles

An assessment of large language models for OpenMP-based code parallelization: a user perspective

Large language models have sparked a lot of attention in the research community in recent days, especially with the introduction of practical tools such as ChatGPT and Github Copilot. Their ability to solve complex programming tasks was also shown in several studies and commercial solutions increasing the interest in using them for software development in different fields. High performance computing is one of such fields, where parallel programming techniques have been extensively used to utilize raw computing power available in contemporary multicore and manycore processors. In this paper, we perform an evaluation of the ChatGPT and Github Copilot tools for OpenMP-based code parallelization using a proposed methodology. We used nine different benchmark applications which represent typical parallel programming workloads and compared their OpenMP-based parallel solutions produced manually and using ChatGPT and Github Copilot in terms of obtained speedup, applied optimizations, and quality of the solution. ChatGPT 3.5 and Github Copilot installed with Visual Studio Code 1.88 were used. We concluded that both tools can produce correct parallel code in most cases. However, performance-wise, ChatGPT can match manually produced and optimized parallel code only in simpler cases, as it lacks a deeper understanding of the code and the context. The results are much better with Github Copilot, where much less effort is needed to obtain correct and performant parallel solution.

Investigating the applications and analysis of physics engine technologies

This research project delves into the performance impact of implementing parallel programming techniques in physics engine applications. With the advent of multi-core processors in contemporary computing environments, optimizing physics simulations through parallel programming has become increasingly feasible. A conventional blob collision physics engine serves as the benchmark for evaluation, and its performance is juxtaposed against a parallel-programmed variant. Experimental findings indicate a significant reduction in computational time required for collision detection and response when parallel processing is employed. This efficiency gain is particularly pronounced in scenarios involving a large number of blobs, showcasing the scalability advantages of parallelization. Moreover, parallel programming facilitates optimal harnessing of multi-core processor capabilities, thereby enhancing the overall efficiency and performance of the physics engine in question. This study not only substantiates the technical merits of applying parallel programming but also illuminates the practical benefits, including resource-efficient operation and quicker simulation times. Consequently, the research provides valuable insights for developers and engineers aiming to fully exploit the capabilities of modern computing hardware in physics-based simulations and applications.

Parallel programming: Driving the computational surge in AI

With the meteoric ascent and ongoing advancements in the realm of Artificial Intelligence (AI), there's an escalating demand for potent computational capabilities. Meeting the exacting computational demands of burgeoning AI technologies has become a pivotal topic in contemporary research. This study is anchored on the theme: "Impact of Parallel Programming on AI." The objective is to delve into the potentialities and value that parallel programming technology might introduce to the AI sector. To deeply probe the integration of parallel programming in AI, we've employed a meticulous research methodology. This process disentangles and scrutinizes the prevalent parallel programming techniques and their tangible applications within the AI sphere. Such a method offers a nuanced grasp of the fusion of parallel programming with AI and the distinct advantages that ensue. Findings reveal a substantial enhancement in the efficacy of AI models when they leverage parallel programming techniques. This is especially salient in scenarios involving large-scale data training and intricate model architectures. Most notably, parallel programming dramatically slashes AI training durations. This swift training paves the way for rapid iterations and fine-tuning. In conclusion, parallel programming emerges as a game-changer in the AI domain. It doesn't merely amplify AI performance but also lays a robust technical bedrock for AI's sustained and swift evolution. This opens up a vista of novel prospects and avenues for forthcoming exploration and application in AI.

Hybrid implementation of the fastICA algorithm for high-density EEG using the capabilities of the Intel architecture and CUDA programming

High-density electroencephalographic (EEG) systems are utilized in the study of the human brain and its underlying behaviors. However, working with EEG data requires a well-cleaned signal, which is often achieved through the use of independent component analysis (ICA) methods. The calculation time for these types of algorithms is the longer the more data we have. This article presents a hybrid implementation of the fastICA algorithm that uses parallel programming techniques (libraries and extensions of the Intel processors and CUDA programming), which results in a significant acceleration of execution time on selected architectures.

Parallelism in a Region Inference Context

Region inference is a type-based program analysis that takes a non-annotated program as input and constructs a program that explicitly manages memory allocation and deallocation by dividing the heap into a stack of regions, each of which can grow and shrink independently from other regions, using constant-time operations. Whereas region-based memory management has shown useful in the contexts of explicit region-based memory management, and in particular, in combination with parallel execution of code, combining region inference with techniques for higher-order parallel programming has not been investigated. In this paper, we present an implementation of a fork-join parallel construct suitable for a compiler based on region inference. We present a minimal higher-order language incorporating the parallel construct, including typing rules and a dynamic semantics for the language, and demonstrate type soundness. We present a novel effect-based region-protection inference algorithm and discuss benefits and shortcomings of the approach. We also describe an efficient implementation embedded in the MLKit Standard ML compiler. Finally, we evaluate the approach and the implementation based on a number of parallel benchmarks, and thereby demonstrate that the technique effectively utilises multi-core architectures in a higher-order functional setting.

Locating the Parameters of RBF Networks Using a Hybrid Particle Swarm Optimization Method

In the present work, an innovative two-phase method is presented for parameter tuning in radial basis function artificial neural networks. These kinds of machine learning models find application in many scientific fields in classification problems or in function regression. In the first phase, a technique based on particle swarm optimization is performed to locate a promising interval of values for the network parameters. Particle swarm optimization was used as it is a highly reliable method for global optimization problems, and in addition, it is one of the fastest and most-flexible techniques of its class. In the second phase, the network was trained within the optimal interval using a global optimization technique such as a genetic algorithm. Furthermore, in order to speed up the training of the network and due to the use of a two-stage method, parallel programming techniques were utilized. The new method was applied to a number of famous classification and regression datasets, and the results were more than promising.

Optimizing Computational Performance with OpenMP Parallel Programming Techniques

The article presents a study of parallel computing, specifically comparing the performance of OpenMP in C++ and Python. Furthermore, the technologies of OpenMP and TPL (C++, C#) are contrasted. Performance indicators were established that showcase the advantages and disadvantages of each methodology. In addition to the numerical data, the research provides insights into the peculiarities of each parallel computing model, which can assist developers in choosing the right tool.

EVALUATION OF THE EFFICIENCY OF THE IMPLEMENTATION OF PARALLEL COMPUTATIONAL ALGORITHMS USING THE <thread> LIBRARY IN C++

Progressive hardware and software mean of paralleling and synchronization of calculations on modern computers with multicore architecture allow to increase the efficiency of computer modeling by increasing (by an order or more) the performance of calculations. The purpose of this work is to increase the efficiency of computational algorithms for the computer implementation of the sweep method by using modern advanced parallel programming techniques. The study used methods of matrix algebra, parallel computations, as well as analysis of the efficiency of algorithms and programs. As a result of the work, computational algorithms for sequential and parallelized in two threads sweep method were developed, and a comparative evaluation of the effectiveness of their implementation by means of thread control library <thread> C++ was performed. The order of SLAE in this case was up to 5×107. As a result of computational experiments, it was possible to achieve an increase in computational speed of 1.88-2.86 times. The results obtained correspond with similar data from available literature sources. The scientific novelty of the work lies in the subsequent development of promising approaches to increase the efficiency of computer simulation through the use of modern technologies and principles of parallel programming with computational experiments on modern hardware and software architectures. For the first time, estimates of the time of software implementation of algorithms for sequential and parallelized by means of the <thread> C++ library computational algorithms for the sweep method for a significant order of SLAE were obtained. The expediency of this paralleling is demonstrated for SLAEs of the order over 2.5×105. The main significance of the work lies in the practical application of the results obtained in computer simulation of engineering problems, the most resource-intensive stage of which is the multiple solution of SLAE of a significant order. Further prospects of research assume in-depth paralleling of algorithms for numerical solution of SLAE by using scalable variations of applied methods, choosing the most productive software technologies, paralleling the program code to the maximum (in terms of the number of processor cores) number of threads.

Implementation of Parallel Programming in One-Dimensional Electromagnetic Simulation with the FDTD Method

Simulation of electromagnetic in 1D has been done using FDTD method. The simulation is conducted on some scenario, starting from one-dimensional free space, absorbing boundary condition in one dimension, propagation in a dielectric medium, simulating different sources, and propagation in a lossy dielectric medium. To accelerate calculation process, parallel programming technique is used on looping parts of simulation.

Global Fixed-Priority Scheduling for Parallel Real-Time Tasks with Constrained Parallelism

With the rapid development of parallel programming techniques and the widespread use of multiprocessors, scheduling and analysis techniques supporting parallel real-time tasks become a critical topic for multiprocessor real-time systems. Global scheduling that allows the vertices of a parallel task to execute on any processor is a promising scheduling approach with guaranteed theoretical bounds and wide use in practice. However, the complex internal structure of parallel tasks leads to extensive inner- and inter-task interference, which leads to significant pessimism in the worst-case timing analysis. In this paper, a global fixed-priority (G-FP) scheduling with constrained parallelism for parallel real-time tasks based on the sporadic directed acyclic graph (DAG) model is proposed. Each DAG task is assigned a parallel threshold, such that the number of processors occupied by the task is limited to the parallel threshold of the task at a time. We propose a heuristic algorithm to set the parallel threshold and present a response-time analysis (RTA) to exploit the feature of the constrained parallel scheduling. Experiments with randomly generated tasks show that the proposed approach improves the schedulability upon G-FP and federated scheduling in terms of acceptance ratio.

Novel Approach to Minimize the Memory Requirements of Frequent Subgraph Mining Techniques

Frequent subgraph mining (FSM) is a subset of the graph mining domain that is extensively used for graph classification and clustering. Over the past decade, many efficient FSM algorithms have been developed with improvements generally focused on reducing the time complexity by changing the algorithm structure or using parallel programming techniques. FSM algorithms also require high memory consumption, which is another problem that should be solved. In this paper, we propose a new approach called Predictive dynamic sized structure packing (PDSSP) to minimize the memory needs of FSM algorithms. Our approach redesigns the internal data structures of FSM algorithms without making algorithmic modifications. PDSSP offers two contributions. The first is the Dynamic Sized Integer Type, a newly designed unsigned integer data type, and the second is a data structure packing technique to change the behavior of the compiler. We examined the effectiveness and efficiency of the PDSSP approach by experimentally embedding it into two state-of-the-art algorithms, gSpan and Gaston. We compared our implementations to the performance of the originals. Nearly all results show that our proposed implementation consumes less memory at each support level, suggesting that PDSSP extensions could save memory, with peak memory usage decreasing up to 38% depending on the dataset.

A New Approach to Minimize Memory Requirements of Frequent Subgraph Mining Algorithms

Frequent subgraph mining (FSM) is a subsection of graph mining domain which is extensively used for graph classification and graph clustering purposes. Over the past decade, many efficient FSM algorithms have been developed. The improvements generally focus on reducing time complexity by changing the algorithm structure or using parallel programming techniques. FSM algorithms have another problem to solve, which is the high memory consumption. In this study, a new approach called Predictive Dynamic Sized Structure Packing (PDSSP) have been proposed to minimize the memory requirement of FSM algorithms. Proposed approach redesigns the internal data structures of FSM algorithms without any algorithmic modifications. PDSSP has two contributions. The first one is the Dynamic Sized Integer Type (ds_Int) which is a newly designed unsigned integer data type. The second contribution is “Data Structure packaging” component that uses a data structure packing technique which changes the behaviour of the compiler. A number of experiments have been conducted to examine the effectiveness and efficiency of the PDSSP approach by embedding it into two state-of-art algorithms called gSpan and Gaston. Proposed implementation have been compared to the official one. Almost all results show that the proposed implementation consumes less memory on each support level. As a result, PDSSP extensions can save memory and the peak memory usage may decrease up to 38% depending on the dataset.

Molint 1.0: A framework for the computation of molecular integrals and their derivatives for density‐fitted methods

AbstractThe efficient computation of molecular integrals and their derivatives is a crucial step in molecular property evaluation in modern quantum chemistry. In this study, an application programming interface (API) framework, denoted Molint, for the computation of molecular integrals and their first derivatives, over contracted Gaussian functions, for the density‐fitted methods is reported. The Molint is free software and it includes overlap, dipole, kinetic, potential, 2‐index, and 3‐index integrals, and their first derivatives. Furthermore, the Molint provides a smooth approach to build the Fock matrix and evaluate analytic gradients for the density‐fitted methods. The Molint is a C++/Fortran hybrid code, which can take advantage of shared‐memory parallel programming techniques. Our results demonstrate that the Molint is an efficient and user‐friendly API for the computation of molecular integrals and their first derivatives.

AAP4All: An Adaptive Auto Parallelization of Serial Code for HPC Systems

High Performance Computing (HPC) technologies are emphasizing to increase the system performance across many disciplines. The primary challenge in HPC systems is how to achieve massive performance by minimum power consumption. However, the modern HPC systems are configured by adding the powerful and energy efficient multi-cores/many-cores parallel computing devices such as GPUs, MIC, and FPGA etc. Due to increasing the complexity of one chip many-cores/multi-cores systems, only well-balanced and optimized parallel programming technique is the solution to provide substantial increase in performance under power consumption limitations. Conventionally, the researchers face various barriers while parallelizing their serial code because they don’t have enough experience to use parallel programming techniques in optimized way. However, to address these obstacles and achieve massive performance under power consumption limitations, we propose an Adaptive and Automatic Parallel programming tool (AAP4All) for both homogeneous and heterogeneous computing systems. A key advantage of proposed tool is an auto recognition of computer system architecture, then translate automatically the input serial C++ code into parallel programming code for that particular detected system. We also evaluate the performance and power consumption while computing the proposed AAP4All model on different computer architectures, and compare the results with existing state-of-the-art parallel programming techniques. Results show that the proposed model increases the system performance substantially by reducing power consumption as well as serial to parallel transformation effort.

Highly Efficient and Scalable Framework for High-Speed Super-Resolution Microscopy

The multiple signal classification algorithm (MUSICAL) is a statistical super-resolution technique for wide-field fluorescence microscopy. Although MUSICAL has several advantages, such as its high resolution, its low computational performance has limited its exploitation. This paper aims to analyze the performance and scalability of MUSICAL for improving its low computational performance. We first optimize MUSICAL for performance analysis by using the latest high-performance computing libraries and parallel programming techniques. Thereafter, we provide insights into MUSICAL's performance bottlenecks. Based on the insights, we develop a new parallel MUSICAL in C++ using Intel Threading Building Blocks and the Intel Math Kernel Library. Our experimental results show that our new parallel MUSICAL achieves a speed-up of up to 30.36x on a commodity machine with 32 cores with an efficiency of 94.88%. The experimental results also show that our new parallel MUSICAL outperforms the previous versions of MUSICAL in Matlab, Java, and Python by 30.43x, 2.63x, and 1.69x, respectively, on commodity machines.

Self-Organized Maps and High-Frequency Image Detail for MRI Image Enhancement

Magnetic resonance imaging (MRI) is a medical technology that uses powerful magnets, radio waves, and a computer to produce images of the body’s internal organs. The patient should be quiet and motionless during the scanning period, as unavoidable movements, such as breathing and heart rate, cause motion artifacts in the image, which cause contrast instability and low-resolution MRI images. Imaging in the clinical setting is performed at low resolution because the scanning time for high-resolution MR imaging would be very long and cumbersome, and imaging is also very expensive. Learning-based image superresolution methods can reconstruct MR images with optimal quality and resolution. However, these methods have problems such as the inability to find the intrinsic relationship between low- and high-resolution image patches in the training dictionary, specification of the proper amount of error in the training and testing stage due to variability in MR image contrast, inability to reconstruct objects by smoothed edges, and use of the backpropagation method in updating their weight vectors. In this paper, we propose a new superresolution method according to competitive learning-based approaches to overcome the problems of previous superresolution methods, which do not have the problems and complexities of those methods. The proposed method includes self-organizing maps as a preprocessor, the nearest neighbor algorithm as a classifier, and a high-frequency filter as a high-frequency image detail extractor. We constructed a single external dictionary from a combination of low-resolution and high-resolution feature patches and trained our SOM network. Next, we reconstruct the high-resolution image by converting the low-resolution input image into feature patch vectors, and for each vector, we find all corresponding neurons in the network and retrieve all their training feature vectors. Next, we train the nearest neighbor algorithm with the recovered vectors plus the input vector and find the best similarity vector to the input vector. After finding all the best similarity vectors to the input vectors, we reconstruct our high superresolution image. The proposed image superresolution method in practical experiments was trained, tested, and evaluated by the Div2k dataset and compared with other traditional and state-of-the-art image enhancement methods on various datasets, such as SET5, SET14, BSDS100, and URBAN100, and presented better results with higher accuracy and quality than traditional and state-of-the-art methods, both visually compared to each other by human and computational benchmarks, such as the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM), to compare image superresolution algorithms. This method is best for reconstructing high-resolution magnetic resonance images that require high-frequency details and sharp edges with a smooth slope of the imaging objects in their structures. The execution time of the proposed algorithm is slower than that of the other algorithms, so we use GPU hardware and parallel programming techniques to increase the algorithm speed.

Standart Dışı Değerlerin Tespitinde DBSCAN Algoritması ile Seri ve Paralel Programlama Performansının Karşılaştırılması

With the introduction of computers into our lives, digital data sizes are increasing gradually. Non-standard values (outliers) which behave differently from the others can be found in these data produced in the digital world. Detection of these values, especially in big data sets; has great importance in fields such as security, insurance, finance, medicine and genetics. Clustering methods of data mining techniques are frequently used in outlier detection in big data sets. Density based DBSCAN (Density-based spatial clustering of applications with noise) algorithm from clustering algorithms which are sensitive to noisy and outlier values is one of the most important methods in outlier detection. In this study, an application was developed using DBSCAN algorithm in C# programming language for the detection of outliers. In the developed application, 2 data sets with different data numbers were examined and analyzed. For the shortest possible data analysis time, serial and parallel programming techniques were used separately. In order to shorten the analysis time of big data sets, parallel class members in TPL (Task Parallel Library) provided with .Net 4.0 were used. In series of analysis of data sets, it was seen that DBSCAN algorithm produces more accurate results and is more practicable than other selected algorithms in terms of outlier detection. When considered in terms of computing performance, parallel programming has become more efficient as the number of data increases.

Free-Surface Effects on the Performance of Flapping-Foil Thruster for Augmenting Ship Propulsion in Waves

Flapping foils located beneath or to the side of the hull of the ship can be used as unsteady thrusters, augmenting ship propulsion in waves. The basic setup is composed of a horizontal wing, which undergoes an induced vertical motion due to the ship’s responses in waves, while the self-pitching motion of the wing is controlled. Flapping foil thrusters can achieve high level of thrust as indicated by measurements and numerical simulations. Due to the relatively small submergence of the above biomimetic ship thrusters, the free-surface effects become significant. In the present work, the effect of the free surface on the performance of flapping foil thruster is assessed by means of two in-house developed computational models. On one hand, a cost-effective time-domain boundary element method (BEM) solver exploiting parallel programming techniques and general purpose programming on graphics processing units (GPGPU) is employed, while on the other hand a higher fidelity RANSE finite volume solver implemented for high performance computing (HPC) is used, and comparative results are presented. BEM and RANSE calculations present quite similar trends with respect to mean submergence depth, presenting 12%, 28%, and 18% of differences concerning the mean values of lift, thrust, and moment coefficients, respectively. The latter differences become very small after enhancement of the BEM model to include viscous corrections. Useful information and data are derived supporting the design of the considered biomimetic thrusters, for moderate submergence depths and conditions characterized by minor flow separation effects.

A low-cost and duplicable portable solar adaptive optics system based on LabVIEW hybrid programming

Abstract We have developed a portable solar adaptive optics (PSAO) for diffraction-limited imaging based on today’s multi-core personal computer. Our PSAO software is written in LabVIEW code, which features block-diagram function based programming and can dramatically speed up the software development. The PSAO can achieve a ~1000 Hz open-loop correction speed with a Shack–Hartmann Wave-front Sensor (SH-WFS) in 11 × 11 sub-aperture configuration. The image shift measurements for solar wave-front sensing are the most time-consuming computations in a solar adaptive optics (AO) system. Since our current LabVIEW program does not fully support multi-core techniques for the image shift measurements, it cannot fully take advantage of the multi-core computer’s power for parallel computation. In order to accelerate the AO system’s running speed, a dedicated message passing interface/open multi-processing parallel programming technique is developed for our LabVIEW-based AO program, which fully supports multi-core parallel computation in LabVIEW programming. Our experiments demonstrate that the hybrid parallel technique can significantly improve the running speed of the solar AO system, and this work paves the way for the applications of a low-cost and duplicable PSAO system for large solar telescopes.

Towards BIMAX: Binary Inclusion-MAXimal Parallel Implementation for Gene Expression Analysis Alejandra

Differential gene expression analysis and clustering techniques have been current tools to study the relation between a gene and biological processes. Since a group of genes may show co-expression under certain conditions, biclustering techniques have been used to find sets of genes sharing similar expression patterns. We present an analysis of the performance of the BIMAX: Binary Inclusion-MAXimal sequential biclustering algorithm. Its performance is evaluated using synthetic datasets. Finally, we propose a strategy of parallelization for optimizing the performance of the BIMAX using parallel programming techniques.