Data Dependence Tests Research Articles

Nonlinear and symbolic data dependence testing

One of the most crucial qualities of an optimizing compiler is its ability to detect when different data references access the same storage location. Such references are said to be data-dependent and they impose constraints on the amount of program modifications the compiler can apply for improving the program's performance. For parallelizing compilers, the most important program constructs to investigate are loops and the array references they contain. In previous work, we have found a serious limitation of current data dependence tests to be that they cannot handle loop bounds or array subscripts that are symbolic, nonlinear expressions. In this paper, we describe a dependence test, called the range test, that can handle such expressions. Briefly, the range test proves independence by determining whether certain symbolic inequalities hold for a permutation of the loop nest. Powerful symbolic analyses and constraint propagation techniques were developed to prove such inequalities. The range test has been implemented in Polaris, a parallelizing compiler developed at the University of Illinois. We present measurements of the range test's performance and compare it with state-of-the-art tests.

Using knowledge-based techniques on loop parallelization for parallelizing compilers

In this paper we propose a knowledge-based approach for solving data dependence testing and loop scheduling problems. A rule-based system, called the K-Test, is developed by repertory grid and attribute ording table to construct the knowledge base. The K-Test chooses an appropriate testing algorithm according to some features of the input program by using knowledge-based techniques, and then applies the resulting test to detect data dependences for loop parallelization. Another rule-based system, called the KPLS, is also proposed to be able to choose an appropriate scheduling by inferring some features of loops and assign parallel loops on multiprocessors for achieving high speedup. The experimental results show that the graceful speedup obtained by our compiler is obvious.

Static and dynamic evaluation of data dependence analysis techniques

Data dependence analysis techniques are the main component of today's trategies for automatic detection of parallelism. Parallelism detection strategies are being incorporated in commercial compilers with increasing frequency because of the widespread use of processors capable of exploiting instruction-level parallelism and the growing importance of multiprocessors. An assessment of the accuracy of data dependence tests is therefore of great importance for compiler writers and researchers. The tests evaluated in this study include the generalized greatest common divisor test, three variants of Banerjee's test, and the Omega test. Their effectiveness was measured with respect to the Perfect Benchmarks and the linear algebra libraries, EISPACK and LAPACK. Two methods were applied, one using only compile-time information for the analysis, and the second using information gathered during program execution. The results indicate that Banerjee's test is for all practical purposes as accurate as the more complex Omega test in detecting parallelism. However, the Omega test is quite effective in proving the existence of dependences, in contrast with Banerjee's test, which can only disprove, or break dependences. The capability of the Omega test of proving dependences could have a significant impact on several compiler algorithms not considered in this study.

The LRPD test

Current parallelizing compilers cannot identify a significant fraction of parallelizable loops because they have complex or statically insufficiently defined access patterns. As parallelizable loops arise frequently in practice, we advocate a novel framework for their identification: speculatively execute the loop as a doall, and apply a fully parallel data dependence test to determine if it had any cross-iteration dependences; if the test fails, then the loop is re-executed serially. Since, from our experience, a significant amount of the available parallelism in Fortran programs can be exploited by loops transformed through privatization and reduction parallelization , our methods can speculatively apply these transformations and then check their validity at run-time. Another important contribution of this paper is a novel method for reduction recognition which goes beyond syntactic pattern matching; it detects at run-time if the values stored in an array participate in a reduction operation, even if they are transferred through private variables and/or are affected by statically unpredictable control flow. We present experimental results on loops from the PERFECT Benchmarks which substantiate our claim that these techniques can yield significant speedups which are often superior to those obtainable by inspector/executor methods.

Effectiveness of data dependence analysis

Data dependence testing is the basic step in detecting loop level parallelism in numerical programs. The problem is undecidable in the general case. Therefore, work has been concentrated on a simplified problem, affine memory disambiguation. In this simpler domain, array references and loops bounds are assumed to be linear integer functions of loop variables. Dataflow information is ignored. For this domain, we have shown that in practice the problem can be solved accurately and efficiently.(1) This paper studies empirically the effectiveness of this domain restriction, how many real references are affine and flow insensitive. We use Larus's llpp system(2) to find all the data dependences dynamically. We compare these to the results given by our affine memory disambiguation system. This system is exact for all the cases we see in practice. We show that while the affine approximation is reasonable, memory disambiguation is not a sufficient approximation for data dependence analysis. We propose extensions to improve the analysis.

Verification of parallel programs

Ensuring the reliability of computer programs is particularly important, especially for systems with critical applications. The program correctness has to be verified to behave as they were intended to do. For parallel machines, the task is even more difficult because results of executing parallel programs can be very unpredictable as many processes are executed at the same time. A programmer is more likely to make mistakes. In this paper, methods to prove the correctness of parallel programs are proposed. They are based on the symbolic execution and the data dependence tests called the Bernstein Methods.

A general data dependence test for dynamic, pointer-based data structures

Optimizing compilers require accurate dependence testing to enable numerous, performance-enhancing transformations. However, data dependence testing is a difficult problem, particularly in the presence of pointers. Though existing approaches work well for pointers to named memory locations (i.e. other variables), they are overly conservative in the case of pointers to unnamed memory locations. The latter occurs in the context of dynamic, pointer-based data structures, used in a variety of applications ranging from system software to computational geometry to N-body and circuit simulations. In this paper we present a new technique for performing more accurate data dependence testing in the presence of dynamic, pointer-based data structures. We will demonstrate its effectiveness by breaking false dependences that existing approaches cannot, and provide results which show that removing these dependences enables significant parallelization of a real application.

Engineering a data dependence test

AbstractMany papers have been written dealing with the science of data dependence tests, particularly for dependence of arrays in loops; the techniques typically reduce the dependence problem to an algebraic problem, then solve the algebraic problem by an algorithm which has some desired blend of efficiency, generality or precision. While a sound theoretical basis is necessary for dependence‐based tools, these papers often leave out many implementation details. This paper addresses the engineering aspects of data dependence testing, particularly focusing on details that are necessary in any competent implementation of a data dependence test.

The direction vector I test

The GCD and Banerjee tests are the standard data dependence tests used to determine whether a loop may be parallelized/vectorized. In an earlier work, (1991) the authors presented a new data dependence test, the I test, which extends the accuracy of the GCD and the Banerjee tests. In the original presentation, only the case of general dependence was considered, i.e., the case of dependence with a direction vector of the form (*,*,...,*). In the present work, the authors generalize the I test to check for data dependence subject to an arbitrary direction vector. >

Beyond induction variables

Induction variable detection is usually closely tied to the strength reduction optimization. This paper studies induction variable analysis from a different perspective, that of finding induction variables for data dependence analysis. While classical induction variable analysis techniques have been used successfully up to now, we have found a simple algorithm based on the Static Single Assignment form of a program that finds all linear induction variables in a loop. Moreover, this algorithm is easily extended to find induction variables in multiple nested loops, to find nonlinear induction variables, and to classify other integer scalar assignments in loops, such as monotonic, periodic and wrap-around variables. Some of these other variables are now classified using ad hoc pattern recognition, while others are not analyzed by current compilers. Giving a unified approach improves the speed of compilers and allows a more general classification scheme. We also show how to use these variables in data dependence testing.

Delinearization

Exact and efficient data dependence testing is a key to success of loop-parallelizing compiler for computationally intensive programs. A number of algorithms has been created to test array references contained in parameter loops for dependence but most of them are unable to answer the following question correctly: Are references C ( i 1 + 10 j 1 ) and C ( i 2 + 5), 0 ≤ i 1 , i 2 ≤ 4, 0 ≤ j 1 , j 2 ≤ 9 independent? The technique introduced in this paper recognizes that i 1 , i 2 and j 1 , j 2 make different order contributions to the subscript index, and breaks dependence equation i 1 + 10 j 1 = i 2 + 10 j 2 + 5 into two equations i 1 = i 2 and 10 j 1 = 10 j 2 which then can be solved independently. Since resulting equations contain less variables it is less expensive to solve them. We call this technique delinearization because it is reverse of the linearization much discussed in the literature. In the introduction we demonstrate that linearized references are used not infrequently in scientific FORTRAN and C codes. Then we present a theorem on which delinearization algorithm is based and the algorithm itself. The algorithm is fairly simple and inexpensive. As a byproduct it tests equations it produces for independence as exactly as it is done by GCD-test and Banerjee inequalities combined. The algorithm has been implemented at Moscow State University in a vectorizer named VIC.

On the perfect accuracy of an approximate subscript analysis test

The Banerjee test is commonly considered to be the more accurate of the two major approximate data dependence tests used in automatic vectorization/parallelization of loops, the other being the GCD test. From its derivation, however, there is no simple explanation of why the Banerjee test should be nearly as accurate as it is given credit for. We present a set of sufficient conditions for the Banerjee test's accuracy, and explain its perceived accuracy in actual practice by proving that under circumstances which occur extremely frequently in actual code, the Banerjee test is, in fact, not approximate, but perfectly accurate.

Program parallelization with interprocedural analysis

In this paper, we review several important issues in interprocedural analysis for parallel computing: call effects, variable aliasing, and execution context. We show the use of traditional summary data flow information in program transformation for parallel computing. We examine three different methods for interprocedural data dependence analysis: (1) linearization; (2) global regions; and (3) atom images. We observe that although useful in some cases, linearization should be avoided when possible, because it often results in less precise data dependence analysis. Global region methods also suffer loss of exact information of subscript coefficients which is required by some commonly used data dependence test algorithms. As a result, an expensive data dependence test must be used. The atom images method is able to overcome this difficulty and is more efficient. Detailed discussions are provided to justify our observation. We also address the issue of alias identification and the propagation of execution context. The problems in array alias identification and a few possible solutions are discussed.

Efficient interprocedural analysis for program parallelization and restructuring

An approach to efficient interprocedural analysis for program parallelization and restructuring is presented. Such analysis is needed for parallelizing loops which contain procedure calls. Our approach captures call effect on data dependencies by propagating the precise information of array subscripts from the called procedure. This allows the optimizing compiler to choose an efficient yet precise data dependence test scheme depending on the complexity of array reference patterns. The other existing methods do not provide such flexibility, hence may suffer from either imprecision or inefficiency. The paper also discusses usage of classical summary information in several important transformations for program parallelization. Experimental results are reported.

Data dependence and its application to parallel processing

Data dependence testing is required to detect parallelism in programs. In this paper data dependence concepts and data dependence direction vectors are reviewed. Data dependence computation in parallel and vector constructs as well as serialdo loops is covered. Several transformations that require data dependence are given as examples, such as vectorization (translating serial code into vector code), concurrentization (translating serial code into concurrent code for multiprocessors), scalarization (translating vector or concurrent code into serial code for a scalar uniprocessor), loop interchanging and loop fusion. The details of data dependence testing including several data dependence decision algorithms are given.

Special issue on parallelism

The articles presented in our Special Issue on parallel processing on the supercomputing scale reflect, to some extent, splits in the community developing these machines. There are several schools ...