Loop Transformation Techniques Research Articles

Roofline-Model-Based Design Space Exploration for Dataflow Techniques of CNN Accelerators

To effectively compute convolutional layers, a complex design space must exist (e.g., the dataflow techniques associated with the layer parameters, loop transformation techniques, and hardware parameters). For efficient design space exploration (DSE) of various dataflow techniques, namely, the weight-stationary ( WS ), output-stationary ( OS ), row-stationary ( RS ), and no local reuse ( NLR ) techniques, the processing element (PE) structure and computational pattern of each dataflow technique are analyzed. Various performance metrics are calculated, namely, the throughput (in giga-operations per second, GOPS), computation-to-communication ratio ( CCR ), on-chip memory usage, and off-chip memory bandwidth, as closed-form expressions of the layer and hardware parameters. In addition, loop interchange and loop unrolling techniques with a double-buffer architecture are assumed. Many roofline model-based simulations are performed to explore relevant dataflow techniques for a wide variety of convolutional layers of typical neural networks. Through simulation, this paper provides insights into the trends in accelerator performance as the layer parameters change. For convolutional layers with large input and output feature map ( ifmap and ofmap ) widths and heights, the GOPS of the NLR dataflow technique tends to be higher than that of the techniques. For convolutional layers with low weight and ofmap widths and heights, the RS dataflow technique achieves optimal GOPS and on-chip memory usage. In the case of convolutional layers with small weight widths and heights, the GOPS of the WS dataflow technique tends to be high. In the case of convolutional layers with small ofmap widths and heights, the OS dataflow technique achieves optimal GOPS and on-chip memory usage.

Mapping Imperfect Loops to Coarse-Grained Reconfigurable Architectures

Nested loops represent a significant portion of application runtime in multimedia and DSP applications, an important domain of applications for coarse-grained reconfigurable architectures (CGRAs). While conventional approaches to mapping nested loops utilize only a single-dimensional pipelining, which is either along the innermost loop or along an outer loop, in this paper, we explore an orthogonal approach of pipelining along multiple loop dimensions by first flattening the loop nest. To remedy the inevitable problem of repetitive outer-loop computation in flattened loops, we present a small set of special operations that can effectively reduce the number and frequency of micro-operations in the pipelined loop. We also present a loop transformation technique that can make our special operations applicable to a broader range of loops, including those with triangular iteration spaces. Our experimental results using imperfect loops from StreamIt benchmarks demonstrate that our special operations can cover a large portion of operations in flattened loops, improve performance of nested loops by nearly 30% over using loop flattening only, and achieve near-ideal executions on CGRAs for imperfect loops.

ON CHIP MEMORY REDUCTION TECHNIQUE FOR DATA DOMINATED EMBEDDED SYSTEMS

This paper proposes an approach for optimization of on-chip memory size in data dominated embedded systems. Large amount of array processing is being involved in this category. In order to produce a cost effective system, efficient designing of memory module is quite critical. The memory module configuration being selected by the designer should be well suitable for the application. In this regard, this paper presents a methodology for effective optimization of on-chip memory. For sensitive applications involving large array processing, the entire processing has to be done using embedded modules. While using such module s, care should be taken to meet optimized profile for the design metrics. With help of loop transformation technique, relatively a good amount of memory size requirement is reduced for the arrays. This approach results in a very close memory estimate and an effective optimization. This methodology can be further extended to meet the high level memory optimization applications based on cache characteristics. Speech processing front end mechanism is implemented and shows that this approach gives up to an achievement 61.3% reduction of overall system memory requirement over the estimation approach. Results are provided in terms of comparison of the two approaches of memory estimation and optimization with respect to both of the program and data segments.

Verification of Loop and Arithmetic Transformations of Array-Intensive Behaviors

Loop transformation techniques along with arithmetic transformations are applied extensively on array and loop intensive behaviors in design of area/energy efficient systems in the domain of multimedia and signal processing applications. Ensuring correctness of such transformations is crucial for the reliability of the designed systems. In this paper, array data dependence graphs (ADDGs) are used to represent both the input and the transformed behaviors and the correctness of the transformations is ensured by proving equivalence of the two ADDGs. A slice-based equivalence checking method of ADDGs is proposed for this purpose. The method relies on the normalization of arithmetic expressions and some simplification rules to handle arithmetic transformations. Unlike many other reported techniques, our method is strong enough to handle several arithmetic transformations along with all kinds of loop transformations. Correctness and complexity of the method have been dealt with. Experimental results on several test cases demonstrate the effectiveness of the method.

Efficient implementation of an SVM-based speech/music classifier by enhancing temporal locality in support vector references

Speech/music classification is an integral part of various consumer electronics applications such as audio codecs, multimedia document indexing, and automatic speech recognition. To achieve high performance at speech/music classification, a support vector machine (SVM) has been widely used as a classifier due to its decent classification capability. However, in order to use an SVM-based speech/music classifier in embedded systems, which gradually replace desktop computer systems, one significant implementation problem needs to be resolved: high implementation cost due to time and energy inefficiency. The memory requirement determined by the dimensionality and the number of support vectors, is generally too high for an embedded systems cache to accommodate resulting in expensive memory accesses. In this paper, two techniques are proposed to reduce expensive memory accesses by enhancing temporal locality in support vector references utilizing fetched data from memory with great efficiency. For this, the patterns in support vector references are first analyzed, and then loop transformation techniques are proposed to improve the temporal locality that register file and cache hierarchy take advantage of. The proposed techniques are evaluated by applying them to a speech codec, and the enhancement is confirmed by measuring the number of memory accesses, overall execution time, and energy consumption.

Optimizing control flow in loops using interval and dependence analysis

We present a novel loop transformation technique, particularly well suited for optimizing embedded compilers, where an increase in compilation time is acceptable in exchange for significant performance increase. The transformation technique optimizes loops containing nested conditional blocks. Specifically, the transformation takes advantage of the fact that the Boolean value of the conditional expression, determining the true/false paths, can be statically analyzed using a novel interval analysis technique that can evaluate conditional expressions in the general polynomial form. Results from interval analysis combined with loop dependency information is used to partition the iteration space of the nested loop. In such cases, the loop nest is decomposed such as to eliminate the conditional test, thus substantially reducing the execution time. Our technique completely eliminates the conditional from the loops (unlike previous techniques) thus further facilitating the application of other optimizations and improving the overall speedup. Applying the proposed transformation technique on loop kernels taken from Mediabench, SPEC-2000, mpeg4, qsdpcm and gimp, on average we measured a 2.34X speedup when running on a UltraSPARC processor, a 2.92X speedup when running on an Intel Core Duo processor, a 2.44X speedup when running on a PowerPC G5 processor and a 2.04X speedup when running on an ARM9 processor. Performance improvement, taking the entire application into account, was also promising: for 3 selected applications (mpeg-enc, mpeg-dec and qsdpcm) we measured 15% speedup on best case (5% on average) for the whole application.

Partial data reuse for nested loop computations: design space exploration for FPGA implementations

Automated compiler analyses and transformation techniques aim at improving design productivity of the mapping process of applications expressed in high-level programming languages to FPGAs. These transformations allow a compiler tool to reduce the number of design cycles and eliminate tedious and error-prone low-level transformations required in this mapping process, while still leading to good designs. Scalar replacement, also known as register promotion, is a very important data-oriented transformation that leads to designs that reduce the number of external memory accesses, and thus reduce execution time, at the expense of storage resource's. In this article we present a combination of loop transformation techniques, namely loop unrolling, loop splitting, and loop interchange with scalar replacement to enable partial data reuse on computations expressed by tightly nested loops pervasive in image processing algorithms. We describe a performance modelling in the presence of partial data reuse. Our experimental results reveal that our model accurately captures the non-trivial execution effects of pipelined implementations in the presence of partial data reuse due to the need to fill-up data buffers. The model thus allows a compiler to explore a large design space with high accuracy, ultimately allowing quickly it to find better designs than designs with limited manual search or brute-force approaches.

Incremental hierarchical memory size estimation for steering of loop transformations

Modern embedded multimedia and telecommunications systems need to store and access huge amounts of data. This becomes a critical factor for the overall energy consumption, area, and performance of the systems. Loop transformations are essential to improve the data access locality and regularity in order to optimally design or utilize a memory hierarchy. However, due to abstract high-level cost functions, current loop transformation steering techniques do not take the memory platform sufficiently into account. They usually also result in only one final transformation solution. On the other hand, the loop transformation search space for real-life applications is huge, especially if the memory platform is still not fully fixed. Use of existing loop transformation techniques will therefore typically lead to suboptimal end-products. It is critical to find all interesting loop transformation instances. This can only be achieved by performing an evaluation of the effect of later design stages at the early loop transformation stage. This article presents a fast incremental hierarchical memory-size requirement estimation technique. It estimates the influence of any given sequence of loop transformation instances on the mapping of application data onto a hierarchical memory platform. As the exact memory platform instantiation is often not yet defined at this high-level design stage, a platform-independent estimation is introduced with a Pareto curve output for each loop transformation instance. Comparison among the Pareto curves helps the designer, or a steering tool, to find all interesting loop transformation instances that might later lead to low-power data mapping for any of the many possible memory hierarchy instances. Initially, the source code is used as input for estimation. However, performing the estimation repeatedly from the source code is too slow for large search space exploration. An incremental approach, based on local updating of the previous result, is therefore used to handle sequences of different loop transformations. Experiments show that the initial approach takes a few seconds, which is two orders of magnitude faster than state-of-the-art solutions but still too costly to be performed interactively many times. The incremental approach typically takes just a few milliseconds, which is another two orders of magnitude faster than the initial approach. This huge speedup allows us for the first time to handle real-life industrial-size applications and get realistic feedback during loop transformation exploration.

Maximize Parallelism Minimize Overhead for Nested Loops via Loop Striping

Majority of scientific and Digital Signal Processing (DSP) applications are recursive or iterative. Transformation techniques are generally applied to increase parallelism for these nested loops. Most of the existing loop transformation techniques either can not achieve maximum parallelism, or can achieve maximum parallelism but with complicated loop bounds and loop indexes calculations. This paper proposes a new technique, loop striping, that can maximize parallelism while maintaining the original row-wise execution sequence with minimum overhead. Loop striping groups iterations into stripes, where all iterations in a stripe are independent and can be executed in parallel. Theorems and efficient algorithms are proposed for loop striping transformations. The experimental results show that loop striping always achieves better iteration period than software pipelining and loop unfolding, improving average iteration period by 50 and 54% respectively.

A scalable embedded JPEG 2000 architecture

The latest image compression standard, JPEG 2000 is well tuned for diverse applications, thus raising various throughput demands on its building blocks. Therefore, a JPEG 2000 encoder with the feature of scalability is favorable for its ability of meeting different throughput requirements. On the other hand, the large amounts of data streams underline the importance of bandwidth optimization in designing the encoder. The initial specification, especially in terms of loop organization and array indices, describes the data manipulations and, subsequently, influences the outcome of the architecture implementation. Therefore, there is a clear need for the exploiting support, and we believe the emphasis should lie on the loop level steering. In this paper, we apply loop transformation techniques to a scalable embedded JPEG 2000 encoder design during the architectural exploration stage, considering not only the balance of throughput among different blocks, but also the reduction of data transfer. The architecture is prototyped onto Xilinx FPGA.

Transforming Complex Loop Nests for Locality

Over the past 20 years, increases in processor speed have dramatically outstripped performance increases for standard memory chips. To bridge this gap, compilers must optimize applications so that data fetched into caches are reused before being displaced. Existing compiler techniques can efficiently optimize simple loop structures such as sequences of perfectly nested loops. However, on more complicated structures, existing techniques are either ineffective or require too much computation time to be practical for a commercial compiler. To optimize complex loop structures both effectively and inexpensively, we present a novel loop transformation, dependence hoisting, for optimizing arbitrarily nested loops, and an efficient framework that applies the new technique to aggressively optimize benchmarks for better locality. Our technique is as inexpensive as the traditional unimodular loop transformation techniques and thus can be incorporated into commercial compilers. In addition, it is highly effective and is able to block several linear algebra kernels containing highly challenging loop structures, in particular, Cholesky, QR, LU factorization without pivoting, and LU with partial pivoting. The automatic blocking of QR and pivoting LU is a notable achievement—to our knowledge, few previous compiler techniques, including theoretically more general loop transformation frameworks [1, 21, 23, 27, 31], were able to completely automate the blocking of these kernels, and none has produced the same blocking as produced by our technique. These results indicate that with low compilation cost, our technique can in practice match the effectiveness of much more expensive frameworks that are theoretically more powerful.

Compiler analysis of irregular memory accesses

Irregular array accesses are array accesses whose array subscripts do not have closed-form expressions in terms of loop indices. Traditional array analysis and loop transformation techniques cannot handle irregular array accesses. In this paper, we study two kinds of simple and common cases of irregular array accesses: single-indexed access and indirect array access. We present techniques to analyze these two cases at compile-time, and we provide experimental results showing the effectiveness of these techniques in finding more implicit loop parallelism at compile-time and improved speedups.

On the stabilizing conic sectors obtained using small-gain techniques

This paper is concerned with the analysis of the scope and properties of the stabilizing conic sectors obtained using small-gain and loop-transformation (SGLT) techniques. First, it is shown how SGLT techniques are able to generate the set of all nonlinear stabilizing controllers by using transformations based on dynamical 2 × 2 block systems (configurations). Then, stabilizing cones obtained from static configurations are considered. They induce only subsets of all stabilizing controllers, however, providing simplification and tractability. For these cones, maximality is characterized in terms of mu values and frequency responses, finally, an algorithm for deriving maximal cones is presented.

Plugging anti and output dependence removal techniques into loop parallelization algorithm

In this paper we shortly survey some loop transformation techniques which break anti or output dependences, or artificial cycles involving such ‘false’ dependences. These false dependences are removed through the introduction of temporary buffer arrays. Next we show how to plug these techniques into loop parallelization algorithms (such as Allen and Kennedy's algorithm). The goal is to extract as many parallel loops as the intrinsic degree of parallelism of the nest authorizes, while avoiding a full memory expansion. We try to reduce the number of temporary arrays that we introduce, as well as their dimension.

Fail-safe stability for dynamic systems using neural-network controllers

The small gain theorem is used to consider the stability of a neural network controlled system under the condition that some of the neurons may fail with attenuated outputs and possibly with the addition of a constant output bias. Sufficient conditions based on the small gain theorem and the circle criterion are obtained for fail-safe stability. A loop-transformation technique is used to overcome the conservative nature of the small gain approach.

Fail-Safe Stability for Neural Network Controlled Systems

The small gain theorem is used to consider the stability of a neural network controlled system under the condition that some of the neurons may fail with outputs fixed at constant levels. A sufficient condition for fail-safe stabilityis obtained. The circle criterion and loop-transformation techniques are used to overcome the conservative nature of the small gain approach.

On loop transformations for generalized cycle shrinking

This paper describes several loop transformation techniques for extracting parallelism from nested loop structures. Nested loops can then be scheduled to run in parallel so that execution time is minimized. One technique is called selective cycle shrinking, and the other is called true dependence cycle shrinking. It is shown how selective shrinking is related to linear scheduling of nested loops and how true dependence shrinking is related to conflict-free mappings of higher dimensional algorithms into lower dimensional processor arrays. Methods are proposed in this paper to find the selective and true dependence shrinkings with minimum total execution time by applying the techniques of finding optimal linear schedules and optimal and conflict-free mappings proposed by W. Shang and A.B. Fortes.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Revisiting cycle shrinking

Several loop transformation techniques have been designed to extract parallelism from nested loop structures. We first review two important approaches, known as Generalized Cycle Shrinking presented by Shang, O'Keefe and Fortes and the Index Shift Method introduced by Liu, Ho and Sheu. The main result of the paper is a new methodology that permits to combine cycle shrinking techniques with the index shift method. We present a new optimization method that produces the best scheduling vector, and we show that we can outperform previous results by an arbitrary speedup factor.