Multithreaded Parallel Programming Research Articles

PyOMP: Multithreaded Parallel Programming in Python

Python is a widely used language in scientific computing. When the goal is high performance, however, Python lags far behind low-level languages such as C and Fortran. To support applications that stress performance, Python needs to access the full capabilities of modern CPUs. That means support for parallel multithreading. In this article, we describe PyOMP, a system that enables OpenMP in Python. Programmers write code in Python with OpenMP, Numba generates code that compiles to LLVM, and the resulting programs run with performance that approaches that from code written with C and OpenMP. In this article, we provide an update on the PyOMP project and explain how to install it and use it to write parallel multithreaded code in Python.

Passive Auto Focusing of Pathological Microscope with Intelligent Field Image Collection Mechanism.

The microscope is one of the widely used pathological equipment to analyze body fluids like blood, sputum, etc. in granular level. In order to reduce workload on pathologists and strengthen the telehealth services, an automatic self-focusing microscope with different field image collection mechanism is required. In this work, the conversion of a compound microscope into a complete digital self-focusing automatic microscope, with intelligent field image collection mechanism, is discussed. This method uses passive autofocusing technique. In this method, most informative regions are identified on the basis of texture information. Features from these identified regions are used for autofocusing the microscope. This system is capable of collecting multiple snaps from different regions of the smear sample slides. The problem with the smear slide is that it has un-uniform thickness upon the glass slide. So some region has a very thick layer and some region has a very thin layer. In general, both of these regions are not considered for pathological analysis. The proposed system is capable to detect the region of smear slide which is suitable for collection of snap images. A soft computing approach is used to detect the desired regions of the sample in the slide. The Raspberry pi is used to design the control section. Multi-threaded parallel programming is used to optimize I/O execution and waiting time. The performance of the proposed system is satisfactory. The average peak signal-to-noise ratio (PSNR) is about 33 in comparison with manual focusing by the domain expert. The performance of the system in terms of computation time, which is calculated on the benchmark microscopic image dataset, is better than other learning-based methods. Autofocusing of pathological microscope with an intelligent field image collection mechanism is highly useful in the remote healthcare domain. This work basically describes a mechanism to migrate the conventional compound microscope into a tale-health service compatible (IoT enabled) microscope. This system is highly suitable for developing countries where an overall change of existing infrastructure is difficult due to economic reasons.

Optimizing Particle Systems through CUDA-Assisted Multithreading

Particle systems present challenges that have warranted and attracted large amount of attention in both usage and optimization. The use of particle systems has driven complexity of simulation to greater needs of data size and accuracy. Optimization, thus, has become a moving target for researchers to reach. Studies show that multithreading has potential to make the simulation efficient while optimizing complex and data-intensive particle systems. The CUDA (Compute Unified Device Architecture) works with programming languages such as C/C++ and Python to make multithreaded parallel programming easier. This work serves to analyze particle systems using CUDA and provide an understanding about how various parameters such as the particle count and grid size influence the simulation performance. We improve the CUDA particles demo by Nvidia using our Python scripts and study the impact of particles and grids on execution time and throughput. Experimental results indicate that a required level of performance can be achieved by varying the number of particles, the size grids, and the orientation of grids as needed.

Automated Bug Detection for High-level Synthesis of Multi-threaded Irregular Applications

Field Programmable Gate Arrays (FPGAs) are becoming an appealing technology in datacenters and High Performance Computing. High-Level Synthesis (HLS) of multi-threaded parallel programs is increasingly used to extract parallelism. Despite great leaps forward in HLS and related debugging methodologies, there is a lack of contributions in automated bug identification for HLS of multi-threaded programs. This work defines a methodology to automatically detect and isolate bugs in parallel circuits generated with HLS. The technique relies on hardware/software Discrepancy Analysis and exploits a pattern-matching algorithm based on Finite State Automata to compare multiple hardware and software threads. Overhead, advantages, and limitations are evaluated on designs generated with an open-source HLS compiler supporting OpenMP.

NoT: a high-level no-threading parallel programming method for heterogeneous systems

Multithreading is the core of mainstream heterogeneous programming methods such as CUDA and OpenCL. However, multithreaded parallel programming requires programmers to handle low-level runtime details, making the programming process complex and error prone. This paper presents no-threading (NoT), a high-level no-threading programming method. It introduces the association structure, a new language construct, to provide a declarative runtime-free expression of different data parallelisms and avoid the use of multithreading. The NoT method designs C-like syntax for the association structure and implements a compiler and runtime system using OpenCL as an intermediate language. We demonstrate the effectiveness of our techniques with multiple benchmarks. The size of the NoT code is comparable to that of the serial code and is far less than that of the benchmark OpenCL code. The compiler generates efficient OpenCL code, yielding a performance competitive with or equivalent to that of the manually optimized benchmark OpenCL code on both a GPU platform and an MIC platform.

Teaching Parallel Programming for Time-Efficient Computer Applications

Academic research and engineering challenge both require high performance computing (HPC), which can be achieved through parallel programming. The existing curricula of most universities do not properly address the major transition from single-core to multicore systems and sequential to parallel programming. They focus on applying application program interface (API) libraries and open multiprocessing (OpenMP), message passing interface (MPI), and compute unified device architecture (CUDA)/GPU techniques. This approach misses the goal of developing students’ long-term ability to solve real-life problems by ‘thinking in parallel’. In this article, a novel approach is proposed to teach parallel computing that will prepare computer application developers for present and future computation challenges. Using multicore/manycore architecture and popular challenging problems from areas like computer science, proposed approach teaches how to analyze and develop efficient solutions for the problems. As preliminary work, some multithreaded parallel programs are introduced to computer science and engineering students. Based on the feedbacks from information technology (IT) professionals and Student Outcomes Assessment Reports, proposed approach has potential to provide adequate knowledge so that students can fulfill the growing industry demands for HPC. Based on the Steady State Heat Equation experiment, CUDA/GPU parallel programming may achieve up to 241x speed up factor while simulating heat transfer on a 5000x5000 thin surface.

Architectural support for thread communications in multi-core processors

In the ongoing quest for greater computational power, efficiently exploiting parallelism is of paramount importance. Architectural trends have shifted from improving single-threaded application performance, often achieved through instruction level parallelism (ILP), to improving multithreaded application performance by supporting thread level parallelism (TLP). Thus, multi-core processors incorporating two or more cores on a single die have become ubiquitous. To achieve concurrent execution on multi-core processors, applications must be explicitly restructured to exploit parallelism, either by programmers or compilers. However, multithreaded parallel programming may introduce overhead due to communications among threads. Though some resources are shared among processor cores, current multi-core processors provide no explicit communications support for multithreaded applications that takes advantage of the proximity between cores. Currently, inter-core communications depend on cache coherence, resulting in demand-based cache line transfers with their inherent latency and overhead. In this paper, we explore two approaches to improve communications support for multithreaded applications. Prepushing is a software controlled data forwarding technique that sends data to destination’s cache before it is needed, eliminating cache misses in the destination’s cache as well as reducing the coherence traffic on the bus. Software Controlled Eviction (SCE) improves thread communications by placing shared data in shared caches so that it can be found in a much closer location than remote caches or main memory. Simulation results show significant performance improvement with the addition of these architecture optimizations to multi-core processors.

Cilk: An Efficient Multithreaded Runtime System

Cilk (pronounced “silk”) is a C-based runtime system for multithreaded parallel programming. In this paper, we document the efficiency of the Cilk work-stealing scheduler, both empirically and analytically. We show that on real and synthetic applications, the “work” and “critical-path length” of a Cilk computation can be used to model performance accurately. Consequently, a Cilk programmer can focus on reducing the computation's work and critical-path length, insulated from load balancing and other runtime scheduling issues. We also prove that for the class of “fully strict” (well-structured) programs, the Cilk scheduler achieves space, time, and communication bounds all within a constant factor of optimal. The Cilk runtime system currently runs on the Connection Machine CM5 MPP, the Intel Paragon MPP, the Sun Sparcstation SMP, and the Cilk-NOW network of workstations. Applications written in Cilk include protein folding, graphic rendering, backtrack search, and the ★Socrates chess program, which won second prize in the 1995 ICCA World Computer Chess Championship.

Cilk

Cilk (pronounced “silk”) is a C-based runtime system for multi-threaded parallel programming. In this paper, we document the efficiency of the Cilk work-stealing scheduler, both empirically and analytically. We show that on real and synthetic applications, the “work” and “critical path” of a Cilk computation can be used to accurately model performance. Consequently, a Cilk programmer can focus on reducing the work and critical path of his computation, insulated from load balancing and other runtime scheduling issues. We also prove that for the class of “fully strict” (well-structured) programs, the Cilk scheduler achieves space, time and communication bounds all within a constant factor of optimal. The Cilk runtime system currently runs on the Connection Machine CM5 MPP, the Intel Paragon MPP, the Silicon Graphics Power Challenge SMP, and the MIT Phish network of workstations. Applications written in Cilk include protein folding, graphic rendering, backtrack search, and the *Socrates chess program, which won third prize in the 1994 ACM International Computer Chess Championship.

Techniques for efficient inline tracing on a shared-memory multiprocessor

While much current research concerns multiprocessor design, few traces of parallel programs are available for analyzing the effect of design trade-offs. Existing trace collection methods have serious drawbacks: trap-driven methods often slow down program execution by more than 1000 times, significantly perturbing program behavior; microcode modification is faster, but the technique is neither general nor portable. This paper describes a new tool, called MPTRACE, for collecting traces of multithreaded parallel programs executing on shared-memory multiprocessors. MPTRACE requires no hardware or microcode modification; it collects complete program traces; it is portable; and it reduces execution-time dilation to less than a factor 3. MPTRACE is based on inline tracing, in which a program is automatically modified to produce trace information as it executes. We show how the use of compiler flow analysis techniques can reduce the amount of data collected and therefore the runtime dilation of the traced program. We also discuss problematic issues concerning buffering and writing of trace data on a multiprocessor.