Parallel Loops Research Articles

Towards automatic binary runtime loop de-parallelization using on-stack replacement

Runtime compilation has opportunities to parallelize code which are generally not available using static parallelization approaches. However, the parallelized code can possibly slowdown the performance due to unforeseen parallel overheads such as synchronization and speculation support pertaining to the chosen parallelization strategy and the underlying parallel platform. Moreover, with the wide usage of heterogeneous architectures, such choice options become more pronounced. In this paper, we consider an adaptive form of the parallelization operation, for the first time. We propose a method for performing on-stack de-parallelization for a parallelized binary loop at runtime, thereby allowing for rapid loop replacement with a more optimized one. For this paper, we consider a loop parallelization strategy and propose a corresponding de-parallelization method. The method relies on stopping the execution at safe points, gathering threads' states, producing a corresponding serial code, and continuing execution serially. The decision to de-parallelize or not is taken based on the anticipated speedup. To assess the extent of our approach, we have conducted an initial study on a small set of programs with various parallelization overheads. Results show up to 4× performance improvement for a synchronization intense program on a 4-core Intel processor.

Parallel loop cluster quantum Monte Carlo simulation of quantum magnets based on global union-find graph algorithm

A large-scale parallel loop cluster quantum Monte Carlo simulation is presented. On 24,576 nodes of the K computer, one loop cluster Monte Carlo update of the world-line configuration of the S=1∕2 antiferromagnetic Heisenberg chain with 2.6×106 spins at inverse temperature 3.1×105 is executed in about 8.62 s, in which global union-find cluster identification on a graph of about 1.1 trillion vertices and edges is performed. By combining the nonlocal global updates and the large-scale parallelization, we have virtually achieved about 1013-fold speed-up from the conventional local update Monte Carlo simulation performed on a single core. We have estimated successfully the antiferromagnetic correlation length and the magnitude of the first excitation gap of the S=4 antiferromagnetic Heisenberg chain for the first time as ξ=1.040(7)×104 and Δ=7.99(5)×10−4, respectively.

HPSM: a programming framework to exploit multi-CPU and multi-GPU systems simultaneously

This paper presents a high-level C++ framework to explore multi-CPU and multi-GPU systems called HPSM. HPSM enables execution of parallel loops and reductions simultaneously over CPUs and GPUs using three parallel backends: Serial, OpenMP, and StarPU. We analysed HPSM development effort with AXPY program through two standard metrics (NCLOC and ES). In addition, we evaluated performance and energy with three parallel benchmarks: N-Body, Hotspot, and CFD solver. HPSM reduced code effort by up to 56.9% compared to StarPU C interface, although it resulted in 2.5× more lines of code compared to OpenMP. The CPU-GPU combination attained speedup results with Hotspot of up to 92.7× on a X86-based system with four GPUs and up to 108.2× on an IBM POWER8+ system with two GPUs. On both systems, the addition of GPUs improved energy efficiency.

HPSM: a programming framework to exploit multi-CPU and multi-GPU systems simultaneously

This paper presents a high-level C++ framework to explore multi-CPU and multi-GPU systems called HPSM. HPSM enables execution of parallel loops and reductions simultaneously over CPUs and GPUs using three parallel backends: Serial, OpenMP, and StarPU. We analysed HPSM development effort with AXPY program through two standard metrics (NCLOC and ES). In addition, we evaluated performance and energy with three parallel benchmarks: N-Body, Hotspot, and CFD solver. HPSM reduced code effort by up to 56.9% compared to StarPU C interface, although it resulted in 2.5× more lines of code compared to OpenMP. The CPU-GPU combination attained speedup results with Hotspot of up to 92.7× on a X86-based system with four GPUs and up to 108.2× on an IBM POWER8+ system with two GPUs. On both systems, the addition of GPUs improved energy efficiency.

Loop Parallelism Maximization for Multimedia Data Processing in Mobile Vehicular Clouds

Mobile vehicular cloud has become popular with the rapid development of cloud computing and mobile computing. Nested loops are usually the most critical part in multimedia and high performance Digital Signal Processing (DSP) systems which are widely used in vehicular applications and systems. In order to further explore the parallelism in nested loops, we study how to maximize the system performance with considering the energy reduction for applications on Chip Multiprocessor (CMP) architectures. We propose an algorithm Energy-Aware Loop Parallelism Maximization (EALPM) to maximize the system performance with the consideration of energy reduction for applications with multidimensional nested loops. Our experiment shows that using the EALPM algorithm significantly improves both performance and energy consumption on average in comparision to other algorithms.

The application of compound proportional resonant control strategy in multiple PV inverters system

In order to eliminate the effect on the grid current caused by the background harmonic voltage and the reference signal on the grid connected multi-inverter, this paper adopts the double closed-loop feed-forward control strategy. This strategy is based on the inductor voltage and the grid-connected current, and the integrated control strategy of quasi-proportional resonance loop parallel to a specific harmonic compensation loop. Based on the closed-loop model of multiple inverters, the change curves of the transfer function of the two control strategies are compared with the feed-forward control and the composite proportional resonance. The two corresponding control methods are used to analyze the current quality of the multi-inverter impact. Finally, the MATLAB/Simulink simulation model is set up to verify the proposed control strategies. The simulation results show that the proposed method can achieve better tracking of the sinusoidal command signal at the fundamental frequency, and enhance the anti-interference ability of the system at the 3rd, 5th, and 7th harmonic frequency.

Extending NUMA-BTLP Algorithm with Thread Mapping Based on a Communication Tree

The paper presents a Non-Uniform Memory Access (NUMA)-aware compiler optimization for task-level parallel code. The optimization is based on Non-Uniform Memory Access—Balanced Task and Loop Parallelism (NUMA-BTLP) algorithm Ştirb, 2018. The algorithm gets the type of each thread in the source code based on a static analysis of the code. After assigning a type to each thread, NUMA-BTLP Ştirb, 2018 calls NUMA-BTDM mapping algorithm Ştirb, 2016 which uses PThreads routine pthread_setaffinity_np to set the CPU affinities of the threads (i.e., thread-to-core associations) based on their type. The algorithms perform an improve thread mapping for NUMA systems by mapping threads that share data on the same core(s), allowing fast access to L1 cache data. The paper proves that PThreads based task-level parallel code which is optimized by NUMA-BTLP Ştirb, 2018 and NUMA-BTDM Ştirb, 2016 at compile-time, is running time and energy efficiently on NUMA systems. The results show that the energy is optimized with up to 5% at the same execution time for one of the tested real benchmarks and up to 15% for another benchmark running in infinite loop. The algorithms can be used on real-time control systems such as client/server based applications which require efficient access to shared resources. Most often, task parallelism is used in the implementation of the server and loop parallelism is used for the client.

A Reconfigurable Hybrid Low Dropout Voltage Regulator for Wide-Range Power Supply Noise Rejection and Energy-Efficiency Trade-Off

A scan-reconfigurable hybrid low dropout (LDO) macro with four operating modes to support tuning across a wide range of power supply noise rejection and power efficiency is presented. The configurations include: 1) a full-digital LDO; 2) an analog loop in parallel with a digital loop in switched mode control; 3) an analog LDO in parallel with discrete current sources; and 4) discrete current sources with an analog LDO in shunt with the load. Measurements on a 130-nm CMOS test chip show a wide power supply noise rejection range of −9 dB to −34 dB and a corresponding power efficiency range of 87% to 56%.

On the Inductive Coupling Between Two Parallel Thin-Wire Circular Loop Antennas

The magnetic coupling between two circular coils is a classical electromagnetic problem which is of interest in many different fields. While existing analytical models make quite strong assumptions on the relative position and assume a quasi-static approximation, the present contribution aims to provide an analytical full-wave model for the magnetic coupling between two parallel thin-wire circular loop antennas. The proposed model is tested by comparison with the results available in literature under the above-mentioned limitations and by numerical integration showing accuracy and reduced computation time.

Glutamatergic and dopaminergic modulation of cortico-striatal circuits probed by dynamic calcium imaging of networks reconstructed in microfluidic chips

Although the prefrontal cortex and basal ganglia are functionally interconnected by parallel loops, cellular substrates underlying their interaction remain poorly understood. One novel approach for addressing this issue is microfluidics, a methodology which recapitulates several intrinsic and synaptic properties of cortico-subcortical networks. We developed a microfluidic device where cortical neurons projected onto striatal neurons in a separate compartment. We exploited real-time (low-resolution/high-output) calcium imaging to register network dynamics and characterize the response to glutamatergic and dopaminergic agents. Reconstructed cortico-striatal networks revealed the progressive appearance of cortical VGLUT1 clusters on striatal dendrites, correlating with the emergence of spontaneous and synchronous glutamatergic responses of striatal neurons to concurrent cortical stimulation. Striatal exposure to the NMDA receptor GluN2A subunit antagonist TCN201 did not affect network rhythm, whereas the GluN2B subunit antagonist RO256981 significantly decreased striatal activity. Dopamine application or the D2/D3 receptor agonist, quinpirole, decreased cortico-striatal synchrony whereas the D1 receptor agonist, SKF38393, was ineffective. These data show that cortico-striatal networks reconstructed in a microfluidic environment are synchronized and present characteristics close to those of their in situ counterparts. They should prove instructive for deciphering the molecular substrates of CNS disorders and evaluating the actions of novel therapeutic agents.

The Ongoing Evolution of OpenMP

This paper presents an overview of the past, present and future of the OpenMP application programming interface (API). While the API originally specified a small set of directives that guided shared memory fork-join parallelization of loops and program sections, OpenMP now provides a richer set of directives that capture a wide range of parallelization strategies that are not strictly limited to shared memory. As we look toward the future of OpenMP, we immediately see further evolution of the support for that range of parallelization strategies and the addition of direct support for debugging and performance analysis tools. Looking beyond the next major release of the specification of the OpenMP API, we expect the specification eventually to include support for more parallelization strategies and to embrace closer integration into its Fortran, C and, in particular, C++ base languages, which will likely require the API to adopt additional programming abstractions.

Cancellation of harmonic torque disturbance in permanent magnet synchronous motor drives by using an adaptive feedforward controller

The effects of disruptive harmonic torque disturbances on permanent magnet synchronous motor drives originating at the mechanical load or from cogging can be mitigated by producing a counteracting electric torque. The study proposes using an adaptive feedforward controller (AFC) for this purpose. The AFC is designed to operate as an auxiliary loop in parallel with a field-oriented controller, which provides speed control. The AFC input is the motor speed control error. The loop is tuned at the frequency of the harmonic torque and its output is added to the q -axis current reference as a variation current of the same frequency. The main advantage of the proposed method is that it does not require an estimation of the disturbance torque. The theoretical analysis of the scheme is presented in the study. Experimental results are also presented in the study from a motor drive system where the mechanical load produces harmonic torques. The results demonstrate the effectiveness of the AFC in significantly reducing speed oscillations by cancelling the effect of the torque harmonic disturbance.

Experimental research on the unstable performances of parallel external loops in the circulating fluidized bed

Severe pressure fluctuations in parallel external loops were found in the circulating fluidized bed (CFB) boilers recently, so a CFB test rig with two external loops was established to investigate the unstable performance of parallel external loops. A long-length horizontal section was used in the loopseal to intensify the impact of gas flow inside horizontal section on solid circulation rate, and CO2 tracer method was used to detect the variation of gas flow rate in the horizontal section. The results showed that the solid circulation rate would significantly influence the performance of parallel external loops. The pressure drop in the parallel external loops at low solid circulation rate was relatively stable if aeration air distributed evenly between the two parallel external loops, and the high-frequency pressure fluctuation in the riser was ascribed to the bubble behavior at turbulent fluidization. However, the fluctuation of gas flow rate inside the horizontal section aggravated at high solid circulation rate, leading to the unstable solid recycling rate, as well as the low-frequency and high-amplitude fluctuation of pressure drop in the parallel external loops. For this reason, a single blower is not recommended to use to supply the aeration rate among parallel external loops.

Combined Resonant Controller and Two-Degree-of-Freedom PID Controller for PMSLM Current Harmonics Suppression

This paper presents a new control method to suppress current harmonics for permanent magnet synchronous linear motor (PMSLM) that is applied in the miniature microsecond laser cutting system. In the control method, the resonant–two-degree-of-freedom (R–2DOF) proportional–integral–derivative (PID) controller is proposed by combining a resonant controller and a two-degree-of-freedom (2DOF) PID controller. The current harmonic components are first analyzed. The resonant controller is subsequently added to the current loop in parallel to the traditional PI controller to suppress the current harmonic components. However, with the current harmonics suppression, the resonant controller can result in the overshoot in the current loop response. The 2DOF PID controller is adopted to reduce the overshoot. Thus, an R–2DOF PID controller is developed by combining the resonant controller and 2DOF PID controller. Meanwhile, the stability of the proposed controller is analyzed. Compared with the traditional PID controller and the Kalman filter, the proposed controller not only can suppress the current harmonics but can reduce the overshoot and thrust ripple as well. Finally, the simulation and experimental comparison results confirm the validity of the proposed control algorithm.

Direct solutions of 3-D magnetotelluric fields using edge-based finite element

This paper utilizes the 3-array variation of the compressed sparse row (CSR) format to store upper triangular elements of coefficient matrix, and applies the Intel MKL PARDISO solver to solve large sparse linear systems of equations, when calculating the 3-D magnetotelluric (MT) fields using edge-based finite element (FE). By default, Intel MKL PARDISO stores matrices in RAM, namely in-core (IC) mode, however, it is quite memory consumption. One can specify that Intel MKL PARDISO stores matrices on disk, it is referred to as out-of-core (OOC) mode. But the OOC mode consumes longer time than the IC mode. We compare both time and memory consumption of IC and OOC mode via COMMEMI 3D-1A and 3D-2A models. As shown in the examples, the OOC mode can solve very large problems since the amount of main memory required is significantly reduced, in the acceptable computing time, which makes frequency loop parallelization of 3-D MT modeling possible on a small memory PC.

Polyhedral-Based Dynamic Loop Pipelining for High-Level Synthesis

Loop pipelining is one of the most important optimization methods in high-level synthesis (HLS) for increasing loop parallelism. There has been considerable work on improving loop pipelining, which mainly focuses on optimizing static operation scheduling and parallel memory accesses. Nonetheless, when loops contain complex memory dependencies, current techniques cannot generate high performance pipelines. In this paper, we extend the capability of loop pipelining in HLS to handle loops with uncertain dependencies (i.e., parameterized by an undetermined variable) and/or nonuniform dependencies (i.e., varying between loop iterations). Our optimization allows a pipeline to be statically scheduled without the aforementioned memory dependencies, but an associated controller will change the execution speed of loop iterations at runtime. This allows the augmented pipeline to process each loop iteration as fast as possible without violating memory dependencies. We use a parametric polyhedral analysis to generate the control logic for when to safely run all loop iterations in the pipeline and when to break the pipeline execution to resolve memory conflicts. Our techniques have been prototyped in an automated source-to-source code transformation framework, with Xilinx Vivado HLS, a leading HLS tool, as the RTL generation backend. Over a suite of benchmarks, experiments show that our optimization can implement optimized pipelines at almost the same clock speed as without our transformations, running approximately 3.7– $10{\times }$ faster, with a reasonable resource overhead.

Passive Damping of Rotationally Periodic Structures with Tuned Piezoelectric Inductive Shunt

This paper considers the piezoelectric resistive and inductive RL shunt damping applied to rotationally periodic structures equipped with an array of regularly spaced piezoelectric patches. A method for simplifying the hardware, by reducing the size of the inductors and eliminating the use of synthetic inductors, is described. The paper compares two different ways of using the piezoelectric array: independent loops and parallel loops. It shows that, if a specific mode with n nodal diameters is targeted, mounting 4n piezoelectric patches in two parallel loops is as efficient as mounting them in 4n independent loops, while considerably reducing the demand on the inductors, L, (by 4n2). The method takes advantage of the mode shapes of rotationally periodic structures. The proposed method is validated numerically and experimentally on a rotationally periodic circular plate (nearly axisymmetric). The proposed technique is aimed at turbomachinery applications.

Parallel Tiled Codes Implementing the Smith-Waterman Alignment Algorithm for Two and Three Sequences.

The Smith-Waterman (SW) algorithm explores all the possible alignments between two or more sequences and as a result it returns the optimal local alignment. However, the computational cost of this algorithm is very high, and the exponential growth of computation makes SW unrealistic for searching similarities in large sets of sequences. Fortunately, the dynamic programming kernel of the SW algorithm involves mathematical operations over affine control loops whose iteration space can be represented by the polyhedral model. This allows us to apply polyhedral compilation techniques to optimize the studied SW dense array code. In this article, we present an approach to generate efficient SW implementations for two and three sequences by using the transitive closure of a dependence graph and loop skewing. Generated programs are represented with parallel tiled loop nests, which expose significantly higher performance than that of programs obtained with closely related compilers. The approach is able to tile all loops of original loop nests as opposed to well-known affine transformation techniques. Furthermore, it allows for code optimization of three-sequence alignment. Such a code cannot be generated by means of state-of-the-art automatic optimizing compilers. We demonstrate that an under-approximation of transitive closure (instead of exact transitive closure) can be used to generate valid parallel tiled code. This considerably reduces the computational complexity of the approach. Generated codes were run on cores of a modern Intel multiprocessor and they expose high speedup and good scalability on this platform.

Dissecting sequential programs for parallelization—An approach based on computational units

SummaryWhen trying to parallelize a sequential program, programmers routinely struggle during the first step: finding out which code sections can be made to run in parallel. While identifying such code sections, most of the current parallelism discovery techniques focus on specific language constructs. In contrast, we propose to concentrate on the computations performed by a program. In our approach, a program is treated as a collection of computations communicating with one another using a number of variables. Each computation is represented as a computational unit (CU). A CU contains the inputs and outputs of a computation, and the three phases of a computation are read, compute, and write. Based on the notion of CU, which ensures that the read phase executes before the write phase, we present a unified framework to identify both loop parallelism and task parallelism in sequential programs. We conducted a range of experiments on 23 applications from four different benchmark suites. Our approach accurately identified the parallelization opportunities in benchmark applications based on comparison with their parallel versions. We have also parallelized the opportunities identified by our approach that were not implemented in the parallel versions of the benchmarks and reported the speedup.

Recognition of the optical packet header for two channels utilizing the parallel reservoir computing based on a semiconductor ring laser

In this paper, an all-optical parallel reservoir computing (RC) system with two channels for the optical packet header recognition is proposed and simulated, which is based on a semiconductor ring laser (SRL) with the characteristic of bidirectional light paths. The parallel optical loops are built through the cross-feedback of the bidirectional light paths where every optical loop can independently recognize each injected optical packet header. Two input signals are mapped and recognized simultaneously by training all-optical parallel reservoir, which is attributed to the nonlinear states in the laser. The recognition of optical packet headers for two channels from 4 bits to 32 bits is implemented through the simulation optimizing system parameters and therefore, the optimal recognition error ratio is 0. Since this structure can combine with the wavelength division multiplexing (WDM) optical packet switching network, the wavelength of each channel of optical packet headers for recognition can be different, and a better recognition result can be obtained.