Data Flow Algorithm Research Articles

Aspects of Creating Parallel Programs in Dataflow Programming Paradigm

The imperative programming paradigm is the main one for creating sequential and parallel programs for the vast majority of modern computers, including supercomputers. A feature of the imperative paradigm is the sequence of commands. This feature is an obstacle to the creation of efficient parallel programs, since parallelism is achieved at the expense of additional code. One of the solutions to the problem of overhead for parallel computing is the creation of such a computing model and the architecture of the system that implements it, for which the parallel execution of the algorithm is an immanent property. This model is a dataflow computing model with a dynamically formed context and the architecture of the parallel dataflow computing system "Buran". A complete transition to dataflow systems is hampered, among other things, by the conceptual difference between the dataflow programming paradigm and the imperative one. The article compares these two paradigms. First, parallel data processing is an inherent property of a dataflow program. Second, the dataflow program consists of three elements: a set of initial data, a program code, and a parameterizable distribution function. And third, a conceptually different approach to the algorithmization of the task — the data themselves store information about who should process them (in traditional programs, on the contrary, the command stores information about what data should be processed). The article also presents the structure of a dataflow program and the route for creating a dataflow algorithm. The translation of basic algorithmic constructions (following, branching, loops) is considered on the example of simple problems.

Possibilities of Using Building Information Model Data in Reinforcement Processing Plant

While the AEC industry is moving towards digitalization off-site rebar prefabrication became a common practice. Now most companies use a long-established standard order processing method, where the customer submits 2D paper or PDF-based drawings. Subsequently, the manufacturers are obligated to make additional detailing, redrawing, calculations, and preparation of other required information for manufacturing. Thus, in this typical scenario, there is a great repetition of the same tasks, with the obvious loss of time and increased likelihood of human error. However, improvements can be made by the application of advanced digital production workflow and the use of open BIM standards (e.g., IFC, XML, BVBS). Therefore, this paper presents the typical data flow algorithm in contrast to the automated data flow for reinforcement manufacturing. Further, the two approaches are compared and analyzed based on Multi-Criteria Decision Making (MCDM) methods. The results have shown promising prospects for companies willing to automate their data flow processes by the use of 3D drawings and digital data from the BIM model in their plants.

A Peak Prediction Method for Subflow in Hybrid Data Flow

Subflow prediction is required in resource active elastic scaling, but the existing single flow prediction methods cannot accurately predict the peak variation of subflow in hybrid data flow. These do not consider the correlation between subflows. The difficulty is that it is hard to calculate the correlation between different data flows in hybrid data flow. In order to solve this problem, this paper proposes a new method DCCSPP (subflow peak prediction of hybrid data flow based on delay correlation coefficients) to predict the peak value of hybrid data flow. Firstly, we establish a delay correlation coefficient model based on the sliding time window to determine the delay time and delay correlation coefficient. Next, based on the model, a hybrid data flow subflow peak prediction model and algorithm are established to achieve accurate peak prediction of subflow. Experiments show that our prediction model has achieved better results. Compared with LSTM, our method has decreased the MAE about 18.36% and RMSE 13.50%. Compared with linear regression, MAE and RMSE are decreased by 27.12% and 25.58%, respectively.

Toward a more dependable hybrid analysis of android malware using aspect-oriented programming

The growing threat to user privacy by Android applications (app) has tremendously increased the need for more reliable and accessible analysis techniques. This paper presents AspectDroid11A poster version of this paper appears in CODASPY 2016 (Ali-Gombe et al., 2016).—an offline app-level hybrid analysis system designed to investigate Android applications for possible unwanted activities. It leverages static bytecode instrumentation to weave in analysis routines into an existing application to provide efficient dataflow analysis, detection of resource abuse, and analytics of suspicious behaviors, which are then monitored dynamically at runtime. Unlike operating system or framework dependent approaches, AspectDroid does not require porting from one version of Android to another, nor does it depend on a particular Android runtime, making it a more adaptable and easier to use technique. We evaluate the strength of our dataflow algorithm on 105 apps from the DroidBench corpus, with experimental results demonstrating that AspectDroid can detect tagged data with 94.68% accuracy. Furthermore, we compare and contrast the behavioral patterns in 100 malware samples from the Drebin dataset (Arp et al., 2014) and 100 apps downloaded from Google Play. Our results showed more traces of sensitive data exfiltration, abuse of resources, as well as suspicious use of programming concepts like reflection, native code, and dynamic classes in the malware set than the Google Play apps. In terms of runtime overhead, our experiments indicate AspectDroid can comprehensively log relevant security concerns with an approximate overhead of 1 MB memory and a 5.9% average increase in CPU usage.

Efficient static checker for tainted variable attacks

Tainted flow attacks originate from program inputs maliciously crafted to exploit software vulnerabilities. These attacks are common in server-side scripting languages, such as PHP. In 1997, Ørbæk and Palsberg formalized the problem of detecting these exploits as an instance of type-checking, and gave an O(V3) algorithm to solve it, where V is the number of program variables. A similar algorithm was, ten years later, implemented on the Pixy tool. In this paper we give an O(V2) solution to the same problem. Our solution uses Bodik et al.’s extended Static Single Assignment (e-SSA) program representation. The e-SSA form can be efficiently computed and it enables us to solve the problem via a sparse dataflow analysis. Using the same infrastructure, we compared a state-of-the-art dataflow solution with our technique. Both approaches have detected 36 vulnerabilities in well known PHP programs. Our results show that our approach tends to outperform the dataflow algorithm for larger inputs. We have reported the new bugs that we found, and an implementation of our algorithm is publicly available at https://github.com/rimsa/tainted-phc.git.

Improved L7-Filter's pattern matching algorithm based on multi-core processors

According to the architecture of multi-core processors and the temporal local characteristics of network data flow,a division and dynamic adaptation algorithm was proposed based on multi-core processors.Classifying network data flow by the type and optimizing chain of rules dynamically by the temporal locality of network flow,the count of the multi-core's L7-Filter matching network data flow were reduced effectively and the processing efficiency was improved dramatically.The simulation result shows that given the number of packets in the same conditions,the algorithm has about 7 percent improvement of the multi-core processing performance.With the increasing number of network packets,the performance superiority becomes more obvious.

Approximate weighted matching on emerging manycore and multithreaded architectures

Graph matching is a prototypical combinatorial problem with many applications in high-performance scientific computing. Optimal algorithms for computing matchings are challenging to parallelize. Approximation algorithms are amenable to parallelization and are therefore important to compute matchings for large-scale problems. Approximation algorithms also generate nearly optimal solutions that are sufficient for many applications. In this paper we present multithreaded algorithms for computing half-approximate weighted matching on state-of-the-art multicore (Intel Nehalem and AMD Magny-Cours), manycore (Nvidia Tesla and Nvidia Fermi), and massively multithreaded (Cray XMT) platforms. We provide two implementations: the first uses shared work queues and is suited for all platforms; and the second implementation, based on dataflow principles, exploits special features available on the Cray XMT. Using a carefully chosen dataset that exhibits characteristics from a wide range of applications, we show scalable performance across different platforms. In particular, for one instance of the input, an R-MAT graph (RMAT-G), we show speedups of about [Formula: see text] on [Formula: see text] cores of an AMD Magny-Cours, [Formula: see text] on [Formula: see text] cores of Intel Nehalem, [Formula: see text] on Nvidia Tesla and [Formula: see text] on Nvidia Fermi relative to one core of Intel Nehalem, and [Formula: see text] on [Formula: see text] processors of Cray XMT. We demonstrate strong as well as weak scaling for graphs with up to a billion edges using up to 12,800 threads. We avoid excessive fine-tuning for each platform and retain the basic structure of the algorithm uniformly across platforms. An exception is the dataflow algorithm designed specifically for the Cray XMT. To the best of the authors' knowledge, this is the first such large-scale study of the half-approximate weighted matching problem on multithreaded platforms. Driven by the critical enabling role of combinatorial algorithms such as matching in scientific computing and the emergence of informatics applications, there is a growing demand to support irregular computations on current and future computing platforms. In this context, we evaluate the capability of emerging multithreaded platforms to tolerate latency induced by irregular memory access patterns, and to support fine-grained parallelism via light-weight synchronization mechanisms. By contrasting the architectural features of these platforms against the Cray XMT, which is specifically designed to support irregular memory-intensive applications, we delineate the impact of these choices on performance.

The Research of AD-HOC Network Bandwidth Control Distribution Mechanism

Ad Hoc network is a novel type of mobile computer network, Its own distinctive quality entrusts with its huge prospects for development. This paper presents a TM program, and thus to determine the proportion of each data stream in multi-hop ad hoc wireless networks accounts for the channel time. At the same time, TM program based on a proposed multi-hop wireless data flow algorithm for fair bandwidth allocation.

Data Flow Analysis as a General Concept for the Transport of Verifiable Program Annotations

Just-in-Time (JIT) compilation is frequently employed in order to speed-up the execution of platform-independent and dynamically extensible mobile code applications. Since the time required for dynamic compilation directly influences a program's execution time, JIT compilers usually utilize only simple and fast techniques for program analysis and optimization. To improve further the analysis and optimization process of such compilers program annotations can be used.However, mostly all current annotation approaches suffer from the fact that the verification of transmitted program information is time consuming and therefore will not be carried out on the consumer side of a mobile code system. In this paper, we present a verifiable annotation technique that is based on a well known iterative data flow algorithm and which can be used for the transmission of all program information that can be derived through data flow analysis. Preliminary measurements of compilation and verification time indicate that the presented technique seems to be implementable and therefore could be used as an all-purpose transportation technique for safe program annotations.

Reconfigurable hardware solution to parallel prefix computation

This paper presents the design and implementation of an efficient reconfigurable parallel prefix computation hardware on field-programmable gate arrays (FPGAs). The design is based on a pipelined dataflow algorithm, and control logic is added to reconfigure the system for arbitrary parallelism degree. The system receives multiple input streams of elements in parallel and produces output streams in parallel. It has an advantage of controlling the degree of parallelism explicitly at run time. The time complexity of the design is O(d+(N?d)/d), where d and N are parallelism degree and stream size, respectively. When the stream size is sufficiently larger than the initial trigger time of the pipeline (d), the time complexity becomes O(N/d). Unlike the prefix computation circuits found in the literature, the design is scalable for different problem sizes including unknown sized data. The design is modular based on a finite state machine, and implemented and tested for target FPGA devices Xilinx Spartan2S XC2S300EFT256-6Q and XC2S600EFG676-6.

Verification method of dataflow algorithms in high-level synthesis

This paper presents a formal verification algorithm using the Petri Net theory to detect design errors for high-level synthesis of dataflow algorithms. Typically, given a dataflow algorithm and a set of architectural constraints, the high-level synthesis performs algorithmic transformation and produces the optimal scheduling. How to verify the correctness of high-level synthesis becomes a key issue before mapping the synthesis results onto a silicon. Many tools exist for RTL (Register Transfer Level) design, but few for high-level synthesis. Instead of applying Boolean algebra, this paper adopts the Petri Net theory to verify the correctness of the synthesis result, because the Petri Net model has the nature of dataflow algorithms. Herein, we propose three approaches to realize the Petri Net based formal verification algorithm and conclude the best one who outperforms the others in terms of processing speed and resource usage.

A conservative algorithm for computing the flow of permissions in Java programs

Open distributed systems are becoming increasingly popular. Such systems include components that may be obtained from a number of different sources. For example, Java allows run-time loading of software components residing on remote machines. One unfortunate side-effect of this openness is the possibility that "hostile" software components may compromise the security of both the program and the system on which it runs. Java offers a built-in security mechanism, using which programmers can give permissions to distributed components and check these permissions at run-time. This security model is flexible, but using it is not straightforward, which may lead to insufficiently tight permission checking and therefore breaches of security.In this paper, we propose a data flow algorithm for automated analysis of the flow of permissions in Java programs. Our algorithm produces, for a given instruction in the program, a set of permissions that are checked on all possible executions up to this instruction. This information can be used in program understanding tools or directly for checking properties that assert what permissions must always be checked before access to certain functionality is allowed. The worst-case complexity of our algorithm is low-order polynomial in the number of program statements and permission types, while comparable previous approaches have exponential costs.

Correctness of dataflow and systolic algorithms using algebras of streams

We present two case studies which illustrate the use of second-order algebra as a formalism for specification and verification of hardware algorithms. In the first case study we specify a systolic algorithm for convolution and formally verify its correctness using second-order equational logic. The second case study demonstrates the expressive power of second-order algebraic specifications by presenting a non-constructive specification of the Hamming stream problem. A dataflow algorithm for computing the Hamming stream is then specified and the correctness of this algorithm is verified by semantical methods. Both case studies illustrate aspects of the metatheory of second-order equational logic.

Compiler analysis for cache coherence: interprocedural array data-flow analysis and its impact on cache performance

In this paper, we present compiler algorithms for detecting references to stale data in shared-memory multiprocessors. The algorithm consists of two key analysis techniques, state reference detection and locality preserving analysis. While the stale reference detection finds the memory reference patterns that may violate cache coherence, the locality preserving analysis minimizes the number of such stale references by analyzing both temporal and spatial reuses. By computing the regions referenced by arrays inside loops, we extend the previous scalar algorithms for more precise analysis. We develop a full interprocedural array data-flow algorithm, which performs both bottom-up side-effect analysis and top-down context analysis on the procedure call graph to further exploit locality across procedure boundaries. The interprocedural algorithm eliminates cache invalidations at procedure boundaries, which were assumed in the previous compiler algorithms. We have fully implemented the algorithm in the Polaris parallelizing compiler. Using execution-driven simulations on Perfect Club benchmarks, we demonstrate how unnecessary cache misses can be eliminated by the automatic stale reference detection. The algorithm can be used to implement cache coherence in the shared-memory multiprocessors that do not have hardware directories, such as Cray T3D.

An efficient algorithm for computing MHP information for concurrent Java programs

Information about which statements in a concurrent program may happen in parallel ( MHP ) has a number of important applications. It can be used in program optimization, debugging, program understanding tools, improving the accuracy of data flow approaches, and detecting synchronization anomalies, such as data races. In this paper we propose a data flow algorithm for computing a conservative estimate of the MHP information for Java programs that has a worst-case time bound that is cubic in the size of the program. We present a preliminary experimental comparison between our algorithm and a reachability analysis algorithm that determines the “ideal” static MHP information for concurrent Java programs. This initial experiment indicates that our data flow algorithm precisely computed the ideal MHP information in the vast majority of cases we examined. In the two out of 29 cases where the MHP algorithm turned out to be less than ideally precise, the number of spurious pairs was small compared to the total number of ideal MHP pairs.

Escape analysis for Java

This paper presents a simple and efficient data flow algorithm for escape analysis of objects in Java programs to determine (i) if an object can be allocated on the stack; (ii) if an object is accessed only by a single thread during its lifetime, so that synchronization operations on that object can be removed. We introduce a new program abstraction for escape analysis, the connection graph , that is used to establish reachability relationships between objects and object references. We show that the connection graph can be summarized for each method such that the same summary information may be used effectively in different calling contexts. We present an interprocedural algorithm that uses the above property to efficiently compute the connection graph and identify the non-escaping objects for methods and threads. The experimental results, from a prototype implementation of our framework in the IBM High Performance Compiler for Java, are very promising. The percentage of objects that may be allocated on the stack exceeds 70% of all dynamically created objects in three out of the ten benchmarks (with a median of 19%), 11% to 92% of all lock operations are eliminated in those ten programs (with a median of 51%), and the overall execution time reduction ranges from 2% to 23% (with a median of 7%) on a 333 MHz PowerPC workstation with 128 MB memory.

A conservative data flow algorithm for detecting all pairs of statements that may happen in parallel

Information about which pairs of statements in a concurrent program can execute in parallel is important for optimizing and debugging programs, for detecting anomalies, and for improving the accuracy of data flow analysis. In this paper, we describe a new data flow algorithm that finds a conservative approximation of the set of all such pairs. We have carried out an initial comparison of the precision of our algorithm and that of the most precise of the earlier approaches, Masticola and Ryder's non-concurrency analysis [8], using a sample of 159 concurrent Ada programs that includes the collection assembled by Masticola and Ryder. For these examples, our algorithm was almost always more precise than non-concurrency analysis, in the sense that the set of pairs identified by our algorithm as possibly happening in parallel is a proper subset of the set identified by non-concurrency analysis. In 132 cases, we were able to use reachability analysis to determine exactly the set of pairs of statements that may happen in parallel. For these cases, there were a total of only 10 pairs identified by our algorithm that cannot actually happen in parallel.

A checkable interface language for pointer-based structures

We present a technique for analysing structural constraints on data aggregates in high-level languages. Our technique includes a formal constraint language and a dataflow algorithm for automatically checking equality constraints. The constraint language is used to augment the type information on program interfaces. For example, one can specify that a procedure must return aggregates A and B where each element in aggregate A points to some element in aggregate B, and that parameter C will have the properties of a rooted tree both on input and output. Our dataflow algorithm tracks the constraints which must apply at each statement in order for the procedure to satisfy its interface, and detects invalid programs which fail to satisfy the constraints on their interfaces. We apply our technique to several examples.Our work is motivated by the requirements for expressive interface definition languages for distributed systems, and by the desire to mechanically check program modules against their interfaces. Our analysis techniques also yield information which may enable compilers and stub generators to produce better implementations.

The combining DAG: a technique for parallel data flow analysis

As the number of available multiprocessors increases, so does the importance of providing software support for these systems, including parallel compilers. Data flow analysis, an important component of software tools, may be computed many times during the compilation of a program, especially when compiling for a multiprocessor. Although converting a sequential data flow algorithm to a parallel algorithm can present some opportunities for computing data flow in parallel, more parallelism can be exposed by the development of new parallel data flow algorithms. We present a technique that computes rapid data flow problems in parallel and thus is applicable for commonly used classical data flow problems, including reaching definitions, reachable uses, available expressions, and very busy expressions. Unlike previous techniques, our technique exploits the inherent parallelism in the data flow computation that occurs across independent paths, within linear paths, and in paths through loops of a control flow graph. The technique first changes cyclic structures in a control flow graph to acyclic structures and then builds a combining directed acyclic graph (DAG) that represents the paths through the control flow graph needed to compute data flow. Data flow is then computed using two passes over the DAG by computing the data flow for the nodes on each level of the DAG in parallel. We also present experimental results comparing the performance of our algorithm with a sequential algorithm and a parallelized sequential algorithm.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An Experimental Study of Three Dataflow Paradigms in Multithreaded Database Transitive Closure Algorithms on Shared Memory Multiprocessors

This paper describes an experimental study of three dataflow paradigms, namely, no dataflow, pipelined dataflow, and network dataflow, in multithreaded database transitive closure algorithms on shared memory multiprocessors. This study shows that dataflow paradigm directly influences performance parameters such as the amount of interthread communication, how data are partitioned among the threads, whether access to each page of data is exclusive or shared, whether locks are needed for concurrency control, and how calculation termination is detected. The algorithm designed with no dataflow outperforms the algorithms with dataflow. Approximately linear speedup is achieved by the no dataflow algorithm with sufficient workload and primary memory. An exclusive access working set model and a shared access working set model describe the interactions between two or more threads′ working sets when access to each page of data is exclusive or shared among the threads, respectively. These models are experimentally verified.