Dynamic Loop Scheduling Research Articles

Design methodology for workload‐aware loop scheduling strategies based on genetic algorithm and simulation

SummaryIn high‐performance computing, the application's workload must be evenly balanced among threads to deliver cutting‐edge performance and scalability. In OpenMP, the load balancing problem arises when scheduling loop iterations to threads. In this context, several scheduling strategies have been proposed, but they do not take into account the input workload of the application and thus turn out to be suboptimal. In this work, we introduce a design methodology to propose, study, and assess the performance of workload‐aware loop scheduling strategies. In this methodology, a genetic algorithm is employed to explore the state space solution of the problem itself and to guide the design of new loop scheduling strategies, and a simulator is used to evaluate their performance. As a proof of concept, we show how the proposed methodology was used to propose and study a new workload‐aware loop scheduling strategy named smart round‐robin (SRR). We implemented this strategy into GNU Compiler Collection's OpenMP runtime. We carry out several experiments to validate the simulator and to evaluate the performance of SRR. Our experimental results show that SRR may deliver up to 37.89% and 14.10% better performance than OpenMP's dynamic loop scheduling strategy in the simulated environment and in a real‐world application kernel, respectively. Copyright © 2016 John Wiley & Sons, Ltd.

Investigating asymptotic properties of vector nonlinear time series models

Analyses and simulations of vector nonlinear time series typically run into weeks or even months because the methods used are computationally intensive. Statisticians have been known to base empirical results on a relatively small number of simulation replications, sacrificing precision and reliability of results in the interest of time and productivity. The simulations are amenable for parallelization. However, parallel computing technology has not yet been widely used in this specific research area. This paper proposes an approach to the parallelization of statistical simulation codes to address the challenge of long running times. Requiring minimal code revision, this approach takes advantage of recent advances in dynamic loop scheduling to achieve high performance on general-purpose clusters, even with the presence of unpredictable load imbalance factors. Preliminary results of applying this approach in the simulation of normal white noise and threshold autoregressive model obtains efficiencies in the range 95%–98% on 8–64 processors. Furthermore, previously unobserved properties of the statistical procedures for the models are uncovered by the simulation.

An Approach of Chunk-based Task Runtime Prediction for Self-Scheduling on Multi-core Desk Grid

Self-Scheduling is a dynamic and adaptive loop scheduling approach to reduce the total execution time for a task running in the cluster or grid environment. This paper focuses on how to use and optimize Self-scheduling technologies to allocate tasks reasonable and achieve better parallel performance. It introduces the prediction algorithms and proposes a novel Chunk-based Task Runtime Prediction (CTRP) algorithm according to the characters of desk grid and multi-core environment. Our experimental results show that our approach can predict the execution time more accurate and achieve better load balancing than that of others when most slave nodes’ load is changing frequently and rulelessly in the multi-core desk grid.

A coarse-grained reconfigurable computing architecture with loop self-pipelining

Reconfigurable computing tries to achieve the balance between high efficiency of custom computing and flexibility of general-purpose computing. This paper presents the implementation techniques in LEAP, a coarse-grained reconfigurable array, and proposes a speculative execution mechanism for dynamic loop scheduling with the goal of one iteration per cycle and implementation techniques to support decoupling synchronization between the token generator and the collector. This paper also introduces the techniques of exploiting both data dependences of intra- and inter-iteration, with the help of two instructions for special data reuses in the loop-carried dependences. The experimental results show that the number of memory accesses reaches on average 3% of an RISC processor simulator with no memory optimization. In a practical image matching application, LEAP architecture achieves about 34 times of speedup in execution cycles, compared with general-purpose processors.

Dynamic load balancing with adaptive factoring methods in scientific applications

To improve the performance of scientific applications with parallel loops, dynamic loop scheduling methods have been proposed. Such methods address performance degradations due to load imbalance caused by predictable phenomena like nonuniform data distribution or algorithmic variance, and unpredictable phenomena such as data access latency or operating system interference. In particular, methods such as factoring, weighted factoring, adaptive weighted factoring, and adaptive factoring have been developed based on a probabilistic analysis of parallel loop iterates with variable running times. These methods have been successfully implemented in a number of applications such as: N-Body and Monte Carlo simulations, computational fluid dynamics, and radar signal processing. The focus of this paper is on adaptive weighted factoring (AWF), a method that was designed for scheduling parallel loops in time-stepping scientific applications. The main contribution of the paper is to relax the time-stepping requirement, a modification that allows the AWF to be used in any application with a parallel loop. The modification further allows the AWF to adapt to load imbalance that may occur during loop execution. Results of experiments to compare the performance of the modified AWF with the performance of the other loop scheduling methods in the context of three nontrivial applications reveal that the performance of the modified method is comparable to, and in some cases, superior to the performance of the most recently introduced adaptive factoring method.

Real-Time Dynamic Voltage Loop Scheduling for Multi-Core Embedded Systems

In this brief, we propose a novel real-time loop-scheduling technique to minimize energy consumption via dynamic voltage scaling (DVS) for applications with loops considering transition overhead. One algorithm, dynamic voltage loop scheduling (DVLS), is designed integrating with DVS. In DVLS, we repeatedly regroup a loop based on rotation scheduling and decrease the energy by DVS as much as possible within a timing constraint. We conduct the experiments on a set of digital signal processing benchmarks. The experimental results show that DVLS achieves big energy saving compared with the traditional time-performance-oriented scheduling algorithm

A Convergence Study of the Discrete FGDLS Algorithm

The Feedback-Guided Dynamic Loop Scheduling (FGDLS) algorithm [1] is a recent dynamic approach to the scheduling of a parallel loop within a sequential outer loop. Earlier papers have analysed convergence under the assumption that the workload is a positive, continuous, function of a continuous argument (the iteration number). However, this assumption is unrealistic since it is known that the iteration number is a discrete variable. In this paper we extend the proof of convergence of the algorithm to the case where the iteration number is treated as a discrete variable. We are able to establish convergence of the FGDLS algorithm for the case when the workload is monotonically decreasing.

Applying the Feedback Guided Dynamic Loop Scheduling Method for the Shortest Path Problem

Feedback Guided Dynamic Loop Scheduling is a recent dynamic method for loop scheduling. The method has been mainly studied for synthetic examples so far and it has always offered the lowest computational times against other dynamic scheduling methods. In this article we show how the method is applied to the shortest path problem and also present some execution times that prove the efficiency.

Design and implementation of a novel dynamic load balancing library for cluster computing

This paper presents the design and implementation of a library based on an integrated approach to dynamic load balancing. This approach combines the advantages of optimizing data migration via novel dynamic loop scheduling strategies with the advances in resource management and task migration capabilities offered by a recently developed parallel runtime system. The performance improvements obtained by the use of this library have been investigated by its use in three scientific applications: the N-body simulations, the profiling of automatic quadrature routines, and the heat solver in an unstructured grid. The experimental results obtained underscore the significance of using such an integrated approach, as well as the benefits of using the library especially in applications characterized by irregular and unpredictable behavior.

Vector Nonlinear Time-Series Analysis of Gamma-Ray Burst Datasets on Heterogeneous Clusters

The simultaneous analysis of a number of related datasets using a single statistical model is an important problem in statistical computing. A parameterized statistical model is to be fitted on multiple datasets and tested for goodness of fit within a fixed analytical framework. Definitive conclusions are hopefully achieved by analyzing the datasets together. This paper proposes a strategy for the efficient execution of this type of analysis on heterogeneous clusters. Based on partitioning processors into groups for efficient communications and a dynamic loop scheduling approach for load balancing, the strategy addresses the variability of the computational loads of the datasets, as well as the unpredictable irregularities of the cluster environment. Results from preliminary tests of using this strategy to fit gamma-ray burst time profiles with vector functional coefficient autoregressive models on 64 processors of a general purpose Linux cluster demonstrate the effectiveness of the strategy.

Feedback Guided Dynamic Loop Scheduling: Convergence of the Continuous Case

In this article we present some theoretical results concerning the convergence of Feedback Guided Dynamic Loop Scheduling (FGDLS). This method was proposed by Bull (Proceedings of Euro-Par'98, Springer-Verlag, 1998) and further developed by Bull, Ford, Freeman and Hancock (Proceedings of Ninth SIAM Conference on Parallel Processing for Scientific Computing, SIAM Press, 1999). Based on several synthetic examples it has been shown that the method performs well when the workload associated with the parallel loop changes relatively slowly (see Bull et al., 1999). However the question of convergence of the FGDLS method has remained an open question. In this paper we establish sufficient conditions for the convergence of the method.

On the Scalability of Dynamic Scheduling Scientific Applications with Adaptive Weighted Factoring

In heterogeneous environments, dynamic scheduling algorithms are a powerful tool towards performance improvement of scientific applications via load balancing. However, these scheduling techniques employ heuristics that require prior knowledge about workload via profiling resulting in higher overhead as problem sizes and number of processors increase. In addition, load imbalance may appear only at run-time, making profiling work tedious and sometimes even obsolete. Recently, the integration of dynamic loop scheduling algorithms into a number of scientific applications has been proven effective. This paper reports on performance improvements obtained by integrating the Adaptive Weighted Factoring, a recently proposed dynamic loop scheduling technique that addresses these concerns, into two scientific applications: computational field simulation on unstructured grids, and N-Body simulations. Reported experimental results confirm the benefits of using this methodology, and emphasize its high potential for future integration into other scientific applications that exhibit substantial performance degradation due to load imbalance.

AN O∥LOG P) PARALLEL IMPLEMENTATION OF FEEDBACK GUIDED DYNAMIC LOOP SCHEDULING

Feedback Guided Dynamic Loop Scheduling (FGDLS) is a recently proposed dynamic algorithm for loop scheduling. The original algorithm required an O(p) serial computation at each stage to compute the updated loop schedule. In this paper, it is shown that this computation can be implemented in O(log p) operations on p processors

A tool for a two-level dynamic load balancing strategy in scientific applications

This paper describes a dynamic load balancing tool intended for computational investigators who have little familiarity with programming for a message-passing environment. Motivated by the PAR DOALL directive available in some compilers for shared-memory systems, the tool is designed to simplify the manual conversion of sequential programs containing computationally intensive one- or two-dimensional loops with independent iterates into parallel programs that execute with high efficiency on general-purpose clusters. The tool implements a dynamic loop scheduling strategy to address load imbalance which may be induced by the non-uniformity of loop iterate times, and by the heterogeneity of processors. The tool is based on the Message Passing Interface library for wide availability. Experimental results of two scientific applications that utilize the tool on a Linux cluster are presented to demonstrate sample achievable performance, and to underscore the effectiveness of the two-level dynamic load balancing strategy.

A LOOP-BASED SCHEDULING ALGORITHM FOR HARDWARE DESCRIPTION LANGUAGES

This paper presents Dynamic Loop Scheduling (DLS), a loop-based algorithm that can efficiently schedule large, control-flow dominated designs. It compares favourably with results produced for traditional path-based approaches and at the same time requires much less overhead to implement. The high-performance of DLS is due mainly to the inclusion of loop feedback edges in the control-flow graph and the interruption of the path generation on the fly. The latter eliminates the generation of false paths thereby avoiding the path explosion problem.

Loop staggering, loop compacting: Restructuring techniques for thrashing problem

Parallel loops account for the greatest amount of parallelism in numerical programs. Executing nested loops in parallel with low run-time overhead is thus very important for achieviog high performance in parallel processing systems. However, in parallel processing systems with caches or local memories in memory hierarchies, “thrashing problem” may arise when data move back and forth frequently between the caches or local memories in different processors. The techniques associated with parallel compiler to solve the problem are not completely developed. In this paper, we present two restructuring techniques called loop staggering, loop staggering and compacting, with which we can not only eliminate the cache or local memory, thrashing phenomena significantly, but also exploit the potential parallelism existing in outer serial loop. Loop staggering benefits the dynamic loop scheduling strategies, whereas loop staggering and compacting is good for static loop scheduling strategies. Our method especially benefits parallel programs, in which a parallel loop is enclosed by a serial loop and array elements are repeatedly used in the different iterations of the parallel loop.