POSIX Threads Research Articles

Stochastic Gradient Descent for matrix completion: Hybrid parallelization on shared- and distributed-memory systems

The purpose of this study is to investigate the hybrid parallelization of the Stochastic Gradient Descent (SGD) algorithm for solving the matrix completion problem on a high-performance computing platform. We propose a hybrid parallel decentralized SGD framework with asynchronous inter-process communication and a novel flexible partitioning scheme to attain scalability up to hundreds of processors. We utilize Message Passing Interface (MPI) for inter-node communication and POSIX threads for intra-node parallelism. We tested our method by using different real-world benchmark datasets. Experimental results on a hybrid parallel architecture showed that, compared to the state-of-the-art, the proposed algorithm achieves 6× higher throughput on sparse datasets, while it achieves comparable throughput on relatively dense datasets.

Seismic Full Waveform Inversion Accelerated by Overlapping Data Input and Computation

Seismic full waveform inversion (FWI) is a powerful technology to obtain high-precision and high-resolution images of subsurface structures. However, FWI is a data-intensive algorithm that needs to read extensive seismic data from disks, which significantly affects its performance. We proposed a portable parallel framework to improve FWI by overlapping data input and computation (ODIC). The framework is based on POSIX threads (Pthreads), which is a standard thread API library and can create a parent thread and a child thread in the FWI process. The former is used to perform computation and the latter to read data from disks, both running simultaneously. This framework has two attractive features. First, it is broadly applicable; it can run on almost any computer from a laptop to a supercomputer. Second, it is easy to implement; it can be readily applied to existing FWI programs. A 3D FWI example shows that the framework speeds up FWI considerably.

Inverse Kinematics for Serial Robot Manipulators by Particle Swarm Optimization and POSIX Threads Implementation

Inverse kinematics is a fundamental problem in manipulator robotics: a set of joint angles must be calculated so that the robot arm can be manipulated to the corresponding desired end effector position and orientation (also known as “pose”). Traditional solution techniques include analytical kinematics solvers, which provide the closed-form expressions for the joint positions as functions of the end-effector pose. When analytical inverse kinematics solvers are not possible due to the manipulator structure, numerical methods such as Newton–Raphson or Jacobian inverse can be used to achieve the task, but at a much slower speed due, to the iterative nature of the computation. Recent swarm intelligence technology has also contributed to manipulator inverse kinematics solutions. In this paper, the use of the Particle Swarm Optimization (PSO) approach in solving the inverse kinematics problem is investigated for the general serial robotic manipulators. Many of the reviewed robotic manipulator inverse kinematics solvers using swarm intelligence only deal with end effector position and not its orientation. Our PSO approach provides the convergence of a complete end-effector pose and will be demonstrated using the Baxter Research Robot, which has two seven-joint arms, although the method is applicable to any general serial robotic manipulator. For computational efficiency, the inverse kinematic calculations were implemented in parallel using Portable Operating Interface (POSIX) threads to take advantage of the independent swarm particle dynamics.

Bounded Verification of Multi-threaded Programs via Lazy Sequentialization

Bounded verification techniques such as bounded model checking (BMC) have successfully been used for many practical program analysis problems, but concurrency still poses a challenge. Here, we describe a new approach to BMC of sequentially consistent imperative programs that use POSIX threads. We first translate the multi-threaded program into a nondeterministic sequential program that preserves reachability for all round-robin schedules with a given bound on the number of rounds. We then reuse existing high-performance BMC tools as backends for the sequential verification problem. Our translation is carefully designed to introduce very small memory overheads and very few sources of nondeterminism, so it produces tight SAT/SMT formulae, and is thus very effective in practice: Our Lazy-CSeq tool implementing this translation for the C programming language won several gold and silver medals in the concurrency category of the Software Verification Competitions (SV-COMP) 2014–2021 and was able to find errors in programs where all other techniques (including testing) failed. In this article, we give a detailed description of our translation and prove its correctness, sketch its implementation using the CSeq framework, and report on a detailed evaluation and comparison of our approach.

Subutai: Speeding Up Legacy Parallel Applications Through Data Synchronization

The decrease of the performance gain dictated by Moore's Law boosted the development of manycore architectures to replace single-core architectures. These new architectures must employ parallel applications and distribute its workload over a multitude of cores to reach the desired performance. Parallel applications are harder to develop than sequential ones since the developer must guarantee data integrity using synchronization primitives. While multiple novel solutions have been proposed to speed up parallel applications through handling one type of data synchronization primitive, exceptionally few works support multiple types of synchronization primitives and legacy code. This article proposes Subutai, a hardware/software co-design solution for accelerating multiple synchronization primitives without modifying the application source code. By providing a new user library, while retaining an existing synchronization API, legacy and novel applications can benefit from our solution. Our experimental evaluation, which provides a POSIX Threads implementation, demonstrates Subutai speeds up to 2.71× and 4.61× the execution of single- and multiple-application executions, respectively.

Analysis of Threading Libraries for High Performance Computing

With the appearance of multi-/many core machines, applications and runtime systems have evolved in order to exploit the new on-node concurrency brought by new software paradigms. POSIX threads (Pthreads) was widely-adopted for that purpose and it remains as the most used threading solution in current hardware. Lightweight thread (LWT) libraries emerged as an alternative offering lighter mechanisms to tackle the massive concurrency of current hardware. In this article, we analyze in detail the most representative threading libraries including Pthread- and LWT-based solutions. In addition, to examine the suitability of LWTs for different use cases, we develop a set of microbenchmarks consisting of OpenMP patterns commonly found in current parallel codes, and we compare the results using threading libraries and OpenMP implementations. Moreover, we study the semantics offered by threading libraries in order to expose the similarities among different LWT application programming interfaces and their advantages over Pthreads. This article exposes that LWT libraries outperform solutions based on operating system threads when tasks and nested parallelism are required.

Sthread: In-Vivo Model Checking of Multithreaded Programs

This work strives to make formal verification of POSIX multithreaded programs easily accessible to general programmers. Sthread operates directly on multithreaded C/C++ programs, without the need for an intermediate formal model. Sthread is in-vivo in that it provides a drop-in replacement for the pthread library, and operates directly on the compiled target executable and application libraries. There is no compiler-generated intermediate representation. The system calls in the application remain unaltered. Optionally, the programmer can add a small amount of additional native C code to include assertions based on the user's algorithm, declarations of shared memory regions, and progress/liveness conditions. The work has two important motivations: (i) It can be used to verify correctness of a concurrent algorithm being implemented with multithreading; and (ii) it can also be used pedagogically to provide immediate feedback to students learning either to employ POSIX threads system calls or to implement multithreaded algorithms. This work represents the first example of in-vivo model checking operating directly on the standard multithreaded executable and its libraries, without the aid of a compiler-generated intermediate representation. Sthread leverages the open-source SimGrid libraries, and will eventually be integrated into SimGrid. Sthread employs a non-preemptive model in which thread context switches occur only at multithreaded system calls (e.g., mutex, semaphore) or before accesses to shared memory regions. The emphasis is on finding "algorithmic bugs" (bugs in an original algorithm, implemented as POSIX threads and shared memory regions. This work is in contrast to Context-Bounded Analysis (CBA), which assumes a preemptive model for threads, and emphasizes implementation bugs such as buffer overruns and write-after-free for memory allocation. In particular, the Sthread in-vivo approach has strong future potential for pedagogy, by providing immediate feedback to students who are first learning the correct use of Pthreads system calls in implementation of concurrent algorithms based on multithreading.

Investigation of heterogeneous computing platforms for real-time data analysis in the CBM experiment

Future experiments in high-energy physics will pose stringent requirements to computing, in particular to real-time data processing. As an example, the CBM experiment at FAIR Germany intends to perform online data selection exclusively in software, without using any hardware trigger, at extreme interaction rates of up to 10 MHz. In this article, we describe how heterogeneous computing platforms, Graphical Processing Units (GPUs) and CPUs, can be used to solve the associated computing problems on the example of the first-level event selection process sensitive to J/ψ decays using muon detectors. We investigate and compare pure parallel computing paradigms (Posix Thread, OpenMP, MPI) and heterogeneous parallel computing paradigms (CUDA, OpenCL) on both CPU and GPU architectures and demonstrate that the problem under consideration can be accommodated with a moderate deployment of hardware resources, provided their compute power is made optimal use of. In addition, we compare OpenCL and pure parallel computing paradigms on CPUs and show that OpenCL can be considered as a single parallel paradigm for all hardware resources.

Accelerating Yade’s poromechanical coupling with matrix factorization reuse, parallel task management, and GPU computing

This study details the acceleration techniques and associated performance gains in the time integration of coupled poromechanical problems using the Discrete Element Method (DEM) and a Pore scale Finite Volume (PFV) scheme in Yade open DEM software. Specifically, the model is tailored for accuracy by reducing the frequency of costly matrix factorizations (matrix factor reuse), moving the matrix factorizations to background POSIX threads (multithreaded factorization), factorizing the matrix on a GPU (accelerated factorization), and running PFV pressure and force calculations in parallel to the DEM interaction loop using OpenMP threads (parallel task management). Findings show that these four acceleration techniques combine to accelerate the numerical poroelastic oedometer solution by 170x, which enables more frequent triangulation of large scale time-dependent DEM+PFV simulations (356 thousand+ particles, 2.1 million DOFs).

Implementation of Improved Synchronization in Inter Process Communication using Threads for Microkernel and Distributed Operating Systems

Interprocess Communication (IPC) is used by the cooperating processes for communication and synchronization. With the advent of Distributed Systems and Microkernel Operating systems, IPC has been used for designing the system for cooperation. This raised the requirements for improving the communication and synchronization for the better performance of the system. Here, a mechanism of synchronization between the processes to reduce the waiting time of process using POSIX (Portable Operating System Interface) threads has been proposed to perform and synchronize the given task.

Combining sequentialization-based verification of multi-threaded C programs with symbolic Partial Order Reduction

Sequentialization has been shown to be an effective symbolic verification technique for safety properties in multi-threaded C programs using POSIX threads. The tool Lazy-CSeq, which applies a lazy sequentialization scheme, demonstrated its efficiency by ranking top places within the concurrency division of the Competitions on Software Verification in recent years. This lazy sequentialization scheme encodes all thread schedules up to a given exploration bound into a single non-deterministic sequential C program. Another particularly effective technique to tackle the state explosion of multi-threaded programs is Partial Order Reduction (POR). It works by limiting the exploration of redundant thread execution orders (i.e., schedules) by exploiting independence. Integrating POR into lazy sequentialization can further significantly improve its scalability. However, a combination of both techniques can lead to unsoundness, i.e., reachability of states is not preserved within the exploration bound. The reason is that POR is not aware of the exploration bound; therefore, POR can prune a schedule which will reach a bug within the exploration bound and instead keep another (functionally) equivalent schedule which requires a larger exploration bound to reach the bug. This paper presents a novel implementation of lazy sequentialization, which integrates symbolic POR into the encoding to prune redundant schedules. An efficient pruning check is employed to make the POR technique aware of the exploration bound and thus avoid unsoundness, while still providing significant state space reductions. On the other hand, the integration of POR entails some additional encoding overhead that can also result in a performance slowdown. Such a case is avoided by a practical heuristic that only employs POR if the encoding overhead of POR is reasonable (i.e., less than 50%). Experimental evaluation shows that our proposed approach can provide considerable speed-ups.

Parallel real-world LU decomposition: Gauss vs. Crout algorithm

Abstract This paper presents numerical experiments with assorted versions of parallel LU matrix decomposition algorithms (Gauss and Crout algorithm). The tests have been carried out on the hardware platform with fourcore Skylake processor featuring hyperthreading technology doubling virtually core number. Parallelization algorithms have been implemented with the aid of classic POSIX threads library. Experiments have shown that basic 4-thread acceleration of all parallel implementations is almost equal to the number of threads/processors. Both algorithms are worth considering in real-world applications (Florida University collection). Gauss algorithm is a better performer, with respect to timing, in the case of matrices with lower density of nonzeros, as opposed to higher density matrices. The latter are processed more efficiently with the aid of Crout algorithm implementation.

Causal set generator and action computer

The causal set approach to quantum gravity has gained traction over the past three decades, but numerical experiments involving causal sets have been limited to relatively small scales. The software suite presented here provides a new framework for the generation and study of causal sets. Its efficiency surpasses previous implementations by several orders of magnitude. We highlight several important features of the code, including the compact data structures, the O(N2) causal set generation process, and several implementations of the O(N3) algorithm to compute the Benincasa–Dowker action of compact regions of spacetime. We show that by tailoring the data structures and algorithms to take advantage of low-level CPU and GPU architecture designs, we are able to increase the efficiency and reduce the amount of required memory significantly. The presented algorithms and their implementations rely on methods that use CUDA, OpenMP, x86 Assembly, SSE/AVX, Pthreads, and MPI. We also analyze the scaling of the algorithms’ running times with respect to the problem size and available resources, with suggestions on how to modify the code for future hardware architectures. Program summaryProgram Title: Causal Set Generator and Action ComputerProgram Files doi:http://dx.doi.org/10.17632/5k8wjrhgwh.1Licensing Provisions: MITProgramming Language: C++/CUDA, x86 AssemblyNature of Problem: Generate causal sets and compute the Benincasa–Dowker action.Solution Method: We generate causal sets sprinkled on a Lorentzian manifold by randomly sampling element coordinates using OpenMP and linking elements using CUDA. Causal sets are stored in a minimal binary representation via the FastBitset class. We measure the action in parallel using OpenMP, SSE/AVX and x86 Assembly. When multiple computers are available, MPI and POSIX threads are also incorporated.Additional Comments: The program runs most efficiently with an Intel processor supporting AVX2 and an NVIDIA GPU with compute capability greater than or equal to 3.0.

Acoustic reverse-time migration using GPU card and POSIX thread based on the adaptive optimal finite-difference scheme and the hybrid absorbing boundary condition

Reverse-time migration (RTM) is a powerful tool for imaging geologically complex structures such as steep-dip and subsalt. However, its implementation is quite computationally expensive. Recently, as a low-cost solution, the graphic processing unit (GPU) was introduced to improve the efficiency of RTM. In the paper, we develop three ameliorative strategies to implement RTM on GPU card. First, given the high accuracy and efficiency of the adaptive optimal finite-difference (FD) method based on least squares (LS) on central processing unit (CPU), we study the optimal LS-based FD method on GPU. Second, we develop the CPU-based hybrid absorbing boundary condition (ABC) to the GPU-based one by addressing two issues of the former when introduced to GPU card: time-consuming and chaotic threads. Third, for large-scale data, the combinatorial strategy for optimal checkpointing and efficient boundary storage is introduced for the trade-off between memory and recomputation. To save the time of communication between host and disk, the portable operating system interface (POSIX) thread is utilized to create the other CPU core at the checkpoints. Applications of the three strategies on GPU with the compute unified device architecture (CUDA) programming language in RTM demonstrate their efficiency and validity.

On the adequacy of lightweight thread approaches for high-level parallel programming models

High-level parallel programming models (PMs) are becoming crucial in order to extract the computational power of current on-node multi-threaded parallelism. The most popular PMs, such as OpenMP or OmpSs, are directive-based: the complexity of the hardware is hidden by the underlying runtime system, improving coding productivity. The implementations of OpenMP usually rely on POSIX threads (pthreads), offering excellent performance for coarse-grained parallelism and a perfect match with the current hardware. OmpSs is a task oriented PM based on an ad hoc runtime solution called Nanos++; it is the precursor of the tasking parallelism in the OpenMP tasking specification. A recent trend in runtimes and applications points to leveraging massive on-node parallelism in conjunction with fine-grained and dynamic scheduling paradigms. In this paper we analyze the behavior of the OpenMP and OmpSs PMs on top of the recently emerged Generic Lightweight Threads (GLT) API. GLT exposes a common API for lightweight thread (LWT) libraries that offers the possibility of running the same application over different native LWT solutions. We describe the design details of those high-level PMs implemented on top of GLT and analyze different scenarios in order to assess where the use of LWTs may benefit application performance. Our work reveals those scenarios where LWTs overperform pthread-based solutions and compares the performance between an ad hoc solution and a generic implementation.

A Scheduling Extension Scheme of the Earliest Deadline First Policy for Hard Real-Time Uniprocessor Systems Integrated on Posix Threads Based on Linux

A Scheduling Extension Scheme of the Earliest Deadline First Policy for Hard Real-Time Uniprocessor Systems Integrated on Posix Threads Based on Linux

Comparative modelling and verification of Pthreads and Dthreads

AbstractThe POSIX threads (Pthreads) library is a thread API for C/C++ to control parallel threads and spawn concurrent process flows. Programming in Pthreads usually suffers from undesirable deadlock, data race, and race condition problems due to the potential nondeterministic execution behaviors between parallel threads. Dthreads, as another multithreading model that re‐implements Pthreads, was proposed by Liu et al for efficient deterministic multithreading. They found out that, under specific test cases, Dthreads can effectively prevent data races. However, no comparison test has been made with Pthreads.To perform a formal comparison between Pthreads and Dthreads over deadlocks, data races, and race conditions, in this paper, we adopt CSP (communicating sequential processes) as a formal model for specifying part of API functions in Pthreads and Dthreads and illustrate the model construction using 4 classical example programs. By feeding the models into the model checker PAT (process analysis toolkit), we have verified that deadlocks and data races exist in Pthreads, but do not exist in Dthreads, for the considered programs. We have also found that neither of them can prevent race conditions. Our comparative modelling and verification of Pthreads and Dthreads show that though Dthreads cannot prevent all the deadlock situations, shown by verification results of another 2 example programs, Dthreads is better than Pthreads on eliminating data races and preventing deadlocks. Considering limited scalability of Dthreads, we have introduced a new programming model to support coarse granularity in bank transfer. Our modelling is also extended by covering the synchronization operations in Liu et al work.

From Pthreads to Multicore Hardware Systems in LegUp High-Level Synthesis for FPGAs

In the last decade, processor speeds have remained fairly stagnant, and to improve performance further, the industry started to increase the number of processor cores. The use of specialized hardware, such as field-programmable gate arrays (FPGAs), has also been on the rise. The traditional design methodology for FPGAs, however, requires hardware knowledge, which makes the platform inaccessible to software engineers. High-level synthesis (HLS) tools aim to resolve this issue by allowing software design methodologies to be used for FPGAs. However, HLS remains difficult to use for many software engineers, as there are tasks, such as system integration, which is still mostly a manual process. Consequently, creating a multicore hardware system on an FPGA is not feasible for most software engineers. To this end, we provide an HLS framework, which can automatically generate a multicore hardware system from software. We provide support for POSIX threads, which can be compiled to concurrently executing hardware cores that can be used in a processor–accelerator hybrid system, or in a hardware-only system without a processor. With this, we show that we can create multicore FPGA systems that can provide significant benefits in performance and energy-efficiency compared with hardware executing sequentially, and software executing on MIPS/ARM/ x86 processors.

Improving the performance of the needleman-wunsch algorithm using parallelization and vectorization techniques

The Needleman-Wunsch (NW) is a dynamic programming algorithm used in the pairwise global alignment of two biological sequences. In this paper, three sets of parallel implementations of the NW algorithm are presented using a mixture of specialized software and hardware solutions: POSIX Threads-based, SIMD Extensions-based and a GPU-based implementations. The three implementations aim at improving the performance of the NW algorithm on large scale input without affecting its accuracy. Our experiments show that the GPU-based implementation is the best implementation as it achieves performance 72.5X faster than the sequential implementation, whereas the best performance achieved by the POSIX threads and the SIMD techniques are 2X and 18.2X faster than the sequential implementation, respectively.

Performance study of a signal‐extraction algorithm using different parallelisation strategies for the Cherenkov Telescope Array's real‐time‐analysis software

SummaryIn this work, a signal‐extraction algorithm pertaining to the Cherenkov Telescope Array's real‐time‐analysis pipeline has been parallelised using SSE, POSIX Threads and CUDA. Because of the observatory's constraints, the online analysis has to be conducted on site, on hardware located at the telescopes, and compels a search for efficient computing solutions to handle the huge amount of measured data. This work is framed in a series of studies which benchmark several algorithms of the real‐time‐analysis pipeline on different architectures to gain an insight into the suitability and performance of each platform.