Data Dependence Tests Research Articles

An Analytical Evaluation of Data Dependence Analysis Techniques

Optimizing compilers rely upon data dependence analysis to reveal the ordering constraints among statements in a program that need to be preserved in order to produce valid optimized and parallel code. Testing array references for data dependence is equivalent to determining whether a system of equalities and inequalities has an integer solution. A number of data dependence tests have been proposed in the literature. In each test there are different tradeoffs between accuracy and efficiency. In this paper we study the fundamental relationships between several data dependence tests. We consider the Banerjee Extreme Value test, Fourier Motzkin Variable Elimination (FMVE), the I-Test, and the Omega test, which are representative of the state of the art in data dependence analysis. The Banerjee Extreme Value test and FMVE can only determine the existence of real solutions to a system. Thus they can only disprove, but not prove, data dependence. The I-Test and the Omega test refine the Banerjee Extreme Value test and FMVE, respectively, to the integer domain and can prove data dependence. The Omega test is a more accurate data dependence test, but with worst case exponential time complexity. The I-Test is a polynomial time test, but it is not always conclusive. We first show that FMVE is equivalent to the Banerjee Extreme Value test. We then show that the Omega test’s technique to refine FMVE to integer solutions (dark shadow) is equivalent to the I-Test’s refinement of the Banerjee Extreme Value test to integer solutions (the accuracy condition). We prove that the I-Test returns an inconclusive (“maybe”) answer if and only if the Omega test requires an exponential time exhaustive search to produce an exact answer (the so-called “Omega Test Nightmare”).

K-DT: a formal system for the evaluation of linear data dependence testing techniques

The power of data dependence testing techniques of a parallelizing compiler is its essence to transform and optimize programs. Numerous techniques were proposed in the past, and it is, however, still a challenging problem to evaluate the relative power of these techniques to better understand the data dependence testing problem. In the past, either empirical studies or experimental evaluation results are published to compare these data dependence testing techniques, being not able to convince the research community completely. In this paper, we show a theoretical study on this issue, comparing the power on disproving dependences of existing techniques by proving theorems in a proposed formal system K-DT. Besides, we also present the upper bounds of these techniques and introduce their minimum complete sets. To the best of our knowledge, K-DT is the first formal system used to compare the power of data dependence testing techniques, and this paper is the first work to show the upper bounds and minimum complete sets of data dependence testing techniques.

The Accuracy of the Non-continuous I Test for One-Dimensional Arrays with References Created by Induction Variables

One-dimensional arrays with subscripts formed by induction variables in real programs appear quite frequently. For most famous data dependence testing methods, checking if integer-valued solutions exist for one-dimensional arrays with references created by induction variable is very difficult. The I test, which is a refined combination of the GCD and Banerjee tests, is an efficient and precise data dependence testing technique to compute if integer-valued solutions exist for one-dimensional arrays with constant bounds and single increments. In this paper, the non-continuous I test, which is an extension of the I test, is proposed to figure out whether there are integer-valued solutions for one-dimensional arrays with constant bounds and non-sing ularincrements or not. Experiments with the benchmarks that have been cited from Livermore and Vector Loop, reveal that there are definitive results for 67 pairs of one-dimensional arrays that were tested.

An improved nonlinear data dependence test

To further parallelize large-scale nonlinear scientific computing applications, some data dependence techniques for nonlinear subscripts, especially for quadratic subscripts, were proposed in the past. The quadratic programming (QP) test and polynomial variable interval (PVI) test are two representative techniques. The QP test, which serves as an exact but time-consuming technique, always gives conservative results when the coefficient matrix of the quadratic terms is not positive semi-definite, while the PVI test will lose efficiency when there exist mixed polynomials in the dependence equation. Focusing on the dependences caused by quadratic subscripts in nonlinear and irregular programs, we propose an improved nonlinear data dependence test in this paper. We first normalize a quadratic equation which is written in a general form, and determine whether the canonical equation is integer solvable in the region of interest based on the interval equation theory. Experimental results show that, compared with the QP test, our method maintains a much lower time complexity. Furthermore, it can detect more general dependences than other dependence testing methods like the PVI test in terms of quadratic subscripts.

A data dependence test based on the projection of paths over shape graphs

We propose a data dependence detection test based on a new conflict analysis algorithm for C codes which make intensive use of recursive data structures dynamically allocated in the heap. This algorithm requires two pieces of information from the code section under analysis (a loop or a recursive function): (i) abstract shape graphs that represent the state of the heap at the code section; and (ii) path expressions that collect the traversing information for each statement. Our algorithm projects the path expressions on the shape graphs and checks over the graphs to ascertain whether one of the sites reached by a write statement matches one of the sites reached by another statement on a different loop iteration (or on a different call instance in a recursive function), in which case a conflict between the two statements is reported. Although our algorithm presents exponential complexity, we have found that in practice the parameters that dominate the computational cost have very low values, and to the best of our knowledge, all the other related studies involve higher costs. In fact, our experimental results show reductions in the data dependence analysis times of one or two orders of magnitude in some of the studied benchmarks when compared to a previous data dependence algorithm. Thanks to the information on uncovered data dependences, we have manually parallelized these codes, achieving speedups of 2.19 to 3.99 in four cores.

An approximate method for filtering out data dependencies with a sufficiently large distance between memory references

In this paper we present an approximate algorithm for detecting and filtering data dependencies with a sufficiently large distance between memory references. A sequence of the same operations (typically enclosed in a `for' loop) can be replaced with a single SIMD operation if the distance between memory references is greater than or equal to the number of data processed in the SIMD register. Some loops that could not be vectorized on traditional vector processors, can still be parallelized for short SIMD execution. There are a number of approximate data-dependence tests that have been proposed in the literature but in all of them data dependency will be assumed when actually there is no such a dependence that could restrict parallelization related to the short SIMD execution model. By examining the properties of linear subscript expressions of possibly conflicting data references, our algorithm gives the green light to the parallelization process if some sufficient conditions regarding the dependence distance are met. Our method is based on the Banerjee test and checks the minimum and maximum distances between memory references within the iteration space rather than searching for the existence of an integer solution to the dependence equation. The proposed method extends the accuracy and applicability of the classical Banerjee test.

Nonlinear Symbolic Analysis for Advanced Program Parallelization

High-end parallel and multicore processors rely on compilers to perform the necessary optimizations and exploit concurrency in order to achieve higher performance. However, the source code for high-performance computers is extremely complex to analyze and optimize. In particular, program analysis techniques often do not take into account complex expressions during the data dependence analysis phase. Most data dependence tests are only able to analyze linear expressions, even though nonlinear expressions occur very often in practice. Therefore, considerable amounts of potential parallelism remain unexploited. In this paper, we propose new data dependence analysis techniques to handle such complex instances of the dependence problem and increase program parallelization. Our method is based on a set of polynomial-time techniques that can prove or disprove dependences in source codes with nonlinear and symbolic expressions, complex loop bounds, arrays with coupled subscripts, and if-statement constraints. In addition, our algorithm can produce accurate and complete direction vector information, enabling the compiler to apply further transformations. To validate our method, we performed an experimental evaluation and comparison against the I-Test, the Omega test, and the Range test in the Perfect and SPEC benchmarks. The experimental results indicate that our dependence analysis tool is accurate, efficient, and more effective in program parallelization than the other dependence tests. The improved parallelization results into higher speedups and better program execution performance in several benchmarks.

Weighting Characteristic Test for Data Dependence Testing on Loops

Weighting Characteristic Test for Data Dependence Testing on Loops

An exact data dependence testing method for quadratic expressions

Optimizing compilers relies on program analysis techniques to detect data dependence between program statements. Data dependence testing is a basic step in detecting loop-level parallelism in numerical program. Most studies indicate that data dependence tests cannot handle nonlinear-expression array subscripts. This study presents an exact dependence test that can handle quadratic expression array subscripts precisely. The proposed method detects whether a quadratic equation is monotonically increasing or decreasing, and then reduces the integer solution interval of each variable by repeated projection. When the effective solution interval for any variable shrinks to empty, no integer solutions exist for this quadratic equation; otherwise, all integer solutions can be found, implying that parallelism of a loop can be exploited.

Detecting loop-carried dependences in programs with dynamic data structures

The problem of data dependences detection in codes based on dynamic data structures, is crucial to various compiler optimizations. The approach presented in this paper focuses on detecting data dependences induced by heap-directed pointers on loops that access dynamic data structures. Knowledge about the shape of the data structure accessible from a heap-directed pointer provides critical information for disambiguating heap accesses originating from it. The new approach is based on a previously developed shape analysis that maintains topological information of the connections among the different nodes (memory locations) in the data structure. As a novelty, our approach carries out abstract interpretation of the statements being analyzed, annotating memory locations with read/write information. This information will be later used in a very accurate data dependence test which we describe in this paper. We also discuss its application to several different benchmarks.

An efficient way to filter out data dependences with a sufficiently large distance between memory references

There are a number of data dependence tests that have been proposed in the literature. The most widely used approximate data dependence tests are the Banerjee inequality and the GCD test. In this paper we consider parallelization for microprocessors with the multimedia extensions. For the short SIMD parallelism extraction it is essential that, if dependency exists, then the distance between memory references is greater than or equal to the number of data processed in the SIMD register. This implies that some loops that could not be vectorized on traditional vector processors can still be parallelized for the short SIMD execution. In this paper we present an accurate and simple method that can filter out data dependences with a sufficiently large distance between memory references for linear array references within a nested loop. The presented method is suitable for use in a dependence analyzer that is organized as a series of tests, progressively increasing in accuracy, as a replacement for the GCD or Banerjee tests.

Toward an automatic parallelization of sparse matrix computations

In this paper, we propose a generic method of automatic parallelization for sparse matrix computation. This method is based on both a refinement of the data-dependence test proposed by Bernstein and an inspector–executor scheme which is specialized to each input program of the compiler. This analysis mixes compilation process and run-time process. The sparsity of underlying data-structure determines a specific parallelism which increases the degree of parallelism of an algorithm. Such a source of parallelism had already been applied to many numerical algorithms such as the usual Cholesky factorization or LU-decomposition algorithms considered as the gold standards of parallelization based on sparsity. The standard automatic parallelization method cannot tackle such source of parallelism because it is based on the value of cells arrays and not merely on the memory addressing function. Addressing the automatization of this parallelism requires to develop a mixed compile-time and runtime approach integrated in a inspector–executor process. The compilation step provides a dedicated inspector devoted to the analyzed program. The inspector computes the dependence graph at runtime which allows a dynamic parallelization of the execution. As expressed just before, the generic scheme developed in this paper follows the design principles which have been applied, but at each time in an ad hoc way, to many sparse parallelization of numerical algorithms such as Cholesky algorithm. As far as we know, no general formal framework has been proposed to automate such a method of sparse parallelization. In this paper, we propose a generic framework of sparse parallelization (i.e. numerical program independent) which can be applied to any numerical programs satisfying the usual syntactic constraints of parallelization.

Data Dependence Analysis Techniques for Increased Accuracy and Extracted Parallelism

Parallelizing compilers rely on data dependence information in order to produce valid parallel code. Traditional data dependence analysis techniques. such as the Banerjee test and the I-Test, can efficiently compute data dependence information for simple instances of the data dependence problem. However, in more complex cases involving triangular or trapezoidal loop regions, symbolic variables, and multidimensional arrays with coupled subscripts these tests, including the triangular Banerjee test, ignore or simplify many of the constraints and thus introduce approximations, especially when testing for data dependence under direction vector constraints. The Omega test can accurately handle such complex cases, but at a higher computation cost. In this paper we extend the ideas behind the I-Test and present new techniques to handle complex instances of tile dependence problem, which are frequently found in actual source code. In particular, we provide polynomial-time techniques that can prove or disprove data dependences, subject to any direction vector, in loops with triangular or trapezoidal bounds, symbolic variables and multidimensional arrays with coupled subscripts. We also investigate the impact of the proposed data dependence analysis techniques in practice. We perform an extensive experimental evaluation of the data dependence analysis tests, including the I-Test, the Omega test and the proposed new techniques. We compare these tests in terms of data dependence accuracy, compilation efficiency and effectiveness in program parallelization. We run several experiments using the Perfect Club benchmarks and the scientific library Lapack. We analyze the trade-off between accuracy and efficiency and the reasons for any approximation of each data dependence test. We determine the impact of the dependence analysis phase on the total compilation time and we measure the number of loops parallelized by each test. We conclude that we can employ, polynomial-time techniques to improve data dependence accuracy and increase program parallelization to a reasonable computation cost.

An experimental evaluation of data dependence analysis techniques

Optimizing compilers rely upon program analysis techniques to detect data dependences between program statements. Data dependence information captures the essential ordering constraints of the statements in a program that need to be preserved in order to produce valid optimized and parallel code. Data dependence testing is very important for automatic parallelization, vectorization, and any other code transformation. In this paper, we examine the impact of data dependence analysis in practice. A number of data dependence tests have been proposed in the literature. In each test, there are different trade offs between accuracy and efficiency. We present an experimental evaluation of several data dependence tests, including the Banerjee test, the I-Test, and the Omega test. We compare these tests in terms of data dependence accuracy, compilation efficiency, effectiveness in parallelization, and program execution performance. We analyze the reasons why a data dependence test can be inexact and we explain how the examined tests handle such cases. We run various experiments using the Perfect Club Benchmarks and the scientific library Lapack. We present the measured accuracy of each test and the reasons for any approximation. We compare these tests in term's of efficiency and we analyze the trade offs between accuracy and efficiency. We also determine the impact of each data dependence test on the total compilation time. Finally, we measure the number of loops parallelized by each test and we compare the execution performance of each benchmark on a multiprocessor. Our results indicate that the Omega test is more accurate, but also very inefficient in the cases where the other two tests are inaccurate. In general, the cost of the Omega test is high and uses a significant percentage of the total compilation time. Furthermore, the difference in accuracy of the Omega test over the Banerjee test and the l-Test does not improve parallelization and program execution performance.

Interprocedural Induction Variable Analysis

Induction variable analysis is an important part of the symbolic analysis in parallelizing compilers. Induction variables can be formed by for or [Formula: see text] loops within procedures or loops of recursive procedure calls. This paper presents an algorithm to find induction variables in formal parameters of procedures caused by recursive procedure calls. The compile-time knowledge of induction variables in formal parameters is essential to summarize array sections to be used for data dependence test and parallelization.

Interactive and symbolic data dependence analysis based on ranges of expressions

Traditional data dependence testing algorithms have become very accurate and efficient for simple subscript expressions, but they cannot handle symbolic expressions because of the complexity of data-flow and lack of the semantic information of variables in programs. In this paper, a range-based testing and query approach, called DDTQ, is proposed to eliminate data dependence between array references with symbolic subscripts. DDTQ firstly extracts data dependence information from the symbolic subscripts, a testing algorithm is then used to disprove the dependence based on the ranges of expressions. The assumed dependence that cannot be handled by the disprover will be converted into simple questions by a question engine so that the compiler can solve them by user interaction in a friendly way. The experiment on perfect benchmarks indicates that DDTQ is effective in improving the parallelizing capability of the compiler.

Dependence analysis with direction vector for array references

We presented a new scheme for exact data dependence analysis, called interval reduction (IR) test in J. Sys. Software [2000;52(1):55–65]. This scheme reduces the solution interval of each constraint variable in the dependence equation by repeated projection. Once the effective solution interval of any constraint variable shrinks to empty, this linear diophantine equation contains no integer solution and the memory accesses subject to this constraint are then independent. Otherwise, all the integer solutions can be enumerated. IR test is an efficient and exact data dependence test method. In the original presentation, only general dependence is considered. If there exists a dependence, we do not care what the direction vector is. In this paper, we extend the IR test to check the data dependence subject to an arbitrary direction vector. Experimental results are presented to show its effectiveness compared to the omega test and power test.

Non-linear array data dependence test

Data dependence analysis is the most essential process while parallelizing a sequential program. Most current data dependence tests cannot handle array subscripts that are non-linear expressions. In this paper, we present a new parallelization algorithm, called non-linear array subscripts (NLA) test, to deal with non-linear or complex array subscripts. In this scheme, the iterations subject to loop-carried dependence are scheduled into different wavefronts, while the iterations with no loop-carried dependence are assigned into the same wavefront. Based on the wavefront information, the original loop is then transformed into parallel code. Our experimental results on shared-memory parallel machines HP SPP2000 and ALR Quad6 prove the high effectiveness of the NLA test.

A simple framework to calculate the reaching definition of array references and its use in subscript array analysis

The property analysis of subscript arrays can be used to facilitate the automatic detection of parallelism in sparse/irregular programs that use indirectly accessed arrays. In order for property analysis to work, array reaching definition information is needed. In this paper, we present a framework to efficiently calculate the array reaching definition. This method is designed to handle the common program patterns in real programs. We use some available techniques as the building components, such as data dependence tests and array summary set representations and operations. Our method is more efficient as well as more flexible than the existing techniques. Copyright © 2000 John Wiley & Sons, Ltd.

The LRPD test: speculative run-time parallelization of loops with privatization and reduction parallelization

Current parallelizing compilers cannot identify a significant fraction of parallelizable loops because they have complex or statically insufficiently defined access patterns. As parallelizable loops arise frequently in practice, we advocate a novel framework for their identification: speculatively execute the loop as a doall and apply a fully parallel data dependence test to determine if it had any cross-iteration dependences; if the test fails, then the loop is reexecuted serially. Since, from our experience, a significant amount of the available parallelism in Fortran programs can be exploited by loops transformed through privatization and reduction parallelization, our methods can speculatively apply these transformations and then check their validity at run-time. Another important contribution of this paper is a novel method for reduction recognition which goes beyond syntactic pattern matching: it detects at run-time if the values stored in an array participate in a reduction operation, even if they are transferred through private variables and/or are affected by statically unpredictable control flow. We present experimental results on loops from the PERFECT Benchmarks, which substantiate our claim that these techniques can yield significant speedups which are often superior to those obtainable by inspector/executor methods.