Parallel Dataflow Research Articles

Spinning fast iterative data flows

Parallel dataflow systems are a central part of most analytic pipelines for big data. The iterative nature of many analysis and machine learning algorithms, however, is still a challenge for current systems. While certain types of bulk iterative algorithms are supported by novel dataflow frameworks, these systems cannot exploit computational dependencies present in many algorithms, such as graph algorithms. As a result, these algorithms are inefficiently executed and have led to specialized systems based on other paradigms, such as message passing or shared memory. We propose a method to integrate incremental iterations , a form of workset iterations, with parallel dataflows. After showing how to integrate bulk iterations into a dataflow system and its optimizer, we present an extension to the programming model for incremental iterations. The extension alleviates for the lack of mutable state in dataflows and allows for exploiting the sparse computational dependencies inherent in many iterative algorithms. The evaluation of a prototypical implementation shows that those aspects lead to up to two orders of magnitude speedup in algorithm runtime, when exploited. In our experiments, the improved dataflow system is highly competitive with specialized systems while maintaining a transparent and unified dataflow abstraction.

Opening the black boxes in data flow optimization

Many systems for big data analytics employ a data flow abstraction to define parallel data processing tasks. In this setting, custom operations expressed as user-defined functions are very common. We address the problem of performing data flow optimization at this level of abstraction, where the semantics of operators are not known. Traditionally, query optimization is applied to queries with known algebraic semantics. In this work, we find that a handful of properties, rather than a full algebraic specification, suffice to establish reordering conditions for data processing operators. We show that these properties can be accurately estimated for black box operators by statically analyzing the general-purpose code of their user-defined functions. We design and implement an optimizer for parallel data flows that does not assume knowledge of semantics or algebraic properties of operators. Our evaluation confirms that the optimizer can apply common rewritings such as selection reordering, bushy join-order enumeration, and limited forms of aggregation push-down, hence yielding similar rewriting power as modern relational DBMS optimizers. Moreover, it can optimize the operator order of nonrelational data flows, a unique feature among today's systems.

HyperFlow and ITK v4 Integration: Exploring the use of a modern parallel dataflow architecture in ITK

In this document, we report our activities on the subcontract with Kitware where we carefully study the upcoming ITK v4 pipeline design to help the ITK v4 development team to add support of modern dataflow architectures to ITK.

Communication-centric optimizations by dynamically detecting collective operations

The steady increase of parallelism in high-performance computing platforms implies that communication will be most important in large-scale applications. In this work, we tackle the problem of transparent optimization of large-scale communication patterns using online compilation techniques. We utilize the Group Operation Assembly Language (GOAL), an abstract parallel dataflow definition language, to specify our transformations in a device-independent manner. We develop fast schemes that analyze dataflow and synchronization semantics in GOAL and detect if parts of the (or the whole) communication pattern express a known collective communication operation. The detection of collective operations allows us to replace the detected patterns with highly optimized algorithms or low-level hardware calls and thus improve performance significantly. Benchmark results suggest that our technique can lead to a performance improvement of orders of magnitude compared with various optimized algorithms written in Co-Array Fortran. Detecting collective operations also improves the programmability of parallel languages in that the user does not have to understand the detailed semantics of high-level communication operations in order to generate efficient and scalable code.

Energy efficient network design tool for green IP/Ethernet networks

An innovative energy-efficient network design algorithm and a tool to lower the network power consumption are proposed. The goal is an energy efficient network that carries its rated communication load while meeting the QoS requirements by using the minimum set of nodes and links. QoS metrics include hop limit, bandwidth limit reliability and stability. The proposed algorithm addresses the set-covered problem. To discover the optimum network configuration efficiently, we propose a network configuration generation algorithm that utilizes DAPDNA-2, a parallel data flow type reconfigurable processer. The processor automatically produces the node/link set in n-digit binary form where links are modeled as “1” for power on and “0” for power off; it also confirms that the QoS requirements are satisfied. Evaluation results show that DAPDNA-2 is 2-orders faster than the conventional sequential method running on a Pentium-4 processor. Prototype Gigabit Layer-2 switches having remote link power on/off and traffic monitoring functions are developed. Using these switches, we successfully demonstrate an energy efficient IP/Ethernet network. The evaluation results show that network power savings of up to 30% can be realized under the NFSNET topology model. The proposed algorithm and power efficient network architecture can be applied to realize the future green network.

Streaming-Enabled Parallel Data Flow Framework in the Visualization ToolKit

A parallel dataflow framework implemented into VTK addresses the need for the streaming parallel computation in visualization pipelines.

Trebuchet: exploring TLP with dataflow virtualisation

Parallel programming has become mandatory to fully exploit the potential of multi-core CPUs. The dataflow model provides a natural way to exploit parallelism. However, specifying dependences and control using fine-grained instructions in dataflow programs can be complex and present unwanted overheads. To address this issue, we have designed TALM: a coarse-grained dataflow execution model to be used on top of widespread architectures. We implemented TALM as the Trebuchet virtual machine for multi-cores. The programmer identifies code blocks that can run in parallel and connects them to form a dataflow graph, which allows one to have the benefits of parallel dataflow execution in a Von Neumann machine, with small programming effort. We parallelised a set of seven applications using our approach and compared with OpenMP implementations. Results show that Trebuchet can be competitive with state-of-the-art technology, while providing the benefits of dataflow execution.

Streaming‐Enabled Parallel Dataflow Architecture for Multicore Systems

AbstractWe propose a new framework design for exploiting multi‐core architectures in the context of visualization dataflow systems. Recent hardware advancements have greatly increased the levels of parallelism available with all indications showing this trend will continue in the future. Existing visualization dataflow systems have attempted to take advantage of these new resources, though they still have a number of limitations when deployed on shared memory multi‐core architectures. Ideally, visualization systems should be built on top of a parallel dataflow scheme that can optimally utilize CPUs and assign resources adaptively to pipeline elements. We propose the design of a flexible dataflow architecture aimed at addressing many of the shortcomings of existing systems including a unified execution model for both demand‐driven and event‐driven models; a resource scheduler that can automatically make decisions on how to allocate computing resources; and support for more general streaming data structures which include unstructured elements. We have implemented our system on top of VTK with backward compatibility. In this paper, we provide evidence of performance improvements on a number of applications.

Multipath routing for eVLBI

Abstract The very long baseline interferometry (VLBI) technique currently demands data storage resources of about 2 TBytes per day which must be analyzed in correlation centers. The current process involves the physical storing and shipping of magnetic disks to these centers, where up to 5 days are needed for transportation. The need of a fast turnaround opened a new research line where all the collected data of observing radiotelescope stations is sent over the Internet. This technique is called eVLBI. Ideally the station is part of a National Research and Education Network (NREN) where multiple intercontinental routes are available. Under this scenario a new protocol has been developed which allows multiple parallel data flows with important throughput improvements. The unique properties of VLBI data imply the development of a custom load control based on user datagram protocol (UDP). A description of the new protocol and performance comparisons of the first demonstrations for eVLBI performed at the Transportable Integrated Geodetic Observatory (TIGO) are included in the present article.

Exploiting global optimizations for openmp programs in the openuh compiler

The advent of new parallel architectures has increased the need for parallel optimizing compilers to assist developers in creating efficient code. OpenUH is a state-of-the-art optimizing compiler, but it only performs a limited set of optimizations for OpenMP programs due to its conservative assumptions of shared memory programming. These limitations may prevent some OpenMP applications from being fully optimized to the extent of its sequential counterpart. This paper describes our design and implementation of a parallel data flow framework, consisting of a Parallel Control Flow Graph (PCFG) and a Parallel SSA (PSSA) representation in OpenUH, to model data flow for OpenMP programs. This framework enables the OpenUH compiler to perform all classical scalar optimizations for OpenMP programs, in addition to conducting OpenMP specific optimizations.

High throughput bandwidth optimized VLSI design for motion compensation in AVS HDTV decoder

In this paper we present a motion compensation (MC) design for the newest Audio Video coding Standard (AVS) of China. Because of compression-efficient techniques of variable block size (VBS) and sub-pixel interpolation, intensive pixel calculation and huge memory access are required. We propose a parallel serial filtering mixed luma interpolation data flow and a three-stage multiplication free chroma interpolation scheme. Compared to the conventional designs, the integrated architecture supports about 2.7 times filtering throughput. The proposed MC design utilizes Vertical Z processing order for reference data re-use and saves up to 30% memory bandwidth. The whole design requires 44.3k gates when synthesized at 108 MHz clock frequency using 0.18-μm CMOS technology and can support up to 1920×1088@30 fps AVS HDTV video decoding.

The evolutionary model of physics large-scale simulation on parallel dataflow architecture

The problem of effective mapping of computational algorithms to parallel architecture is very important in the large-scale simulation. The developed model allows us to explore and utilize fine-grain parallelism, as well as coarse-grain parallelism. The model was tested by the nonlinear 3D magnetohydrodynamic (MHD) code.

PACT XPP—A Self-Reconfigurable Data Processing Architecture

The eXtreme Processing Platform (XPPTM) is a new runtime-reconfigurable data processing architecture. It is based on a hierarchical array of coarsegrain, adaptive computing elements, and a packet-oriented communication network. The strength of the XPPTM technology originates from the combination of array processing with unique, powerful run-time reconfiguration mechanisms. Parts of the array can be configured rapidly in parallel while neighboring computing elements are processing data. Reconfiguration is triggered externally or even by special event signals originating within the array, enabling self-reconfiguring designs. The XPPTM architecture is designed to support different types of parallelism: pipelining, instruction level, data flow, and task level parallelism. Therefore this technology is well suited for applications in multimedia, telecommunications, simulation, signal processing (DSP), graphics, and similar stream-based application domains. The anticipated peak performance of the first commercial device running at 150 MHz is estimated to be 57.6 GigaOps/sec, with a peak I/O bandwidth of several GByte/sec. Simulated applications achieve up to 43.5 GigaOps/sec (32-bit fixed point).

Mixed-Signal Architecture for Real-Time Two-Dimensional Live TV Image Restoration

This paper describes novel electronic architecture for a real-time image restoration of live TV signals at live TV frame rates with a latency of two frames. The spatial filtering is obtained from two electronic analog filters, one for the raster lines and one for the columns. The very fast response of analog filters makes possible truly real-time video frame rate performance. The digital part of the system serves the purpose of pipelined parallel data conversion and flow, rather than that of image processing (filtering). This architecture exhibits some very important advantages. It does not need any computational source, it is very fast, and it is quite inexpensive. In addition, such a “parallel analog computer” can be easily incorporated into any complex system with video signal data as a simple “plug-in” between the camera and monitor. An important aspect is that the system carries low digitalization noise, thus yielding good SNR characteristics. The system can perform any kind of separable spatial frequency filtering and can be electronically tuned to obtain exact performance parameters. Because of these advantages, this architecture is promising for a variety of systems such as supermarket multicamera security, military and aerospace vision systems, and medical diagnostics.

Team CTA Applied to Radar Operations System Modernization

A team Cognitive Task Analysis (CTA) approach has been applied to the modernization of a radar operations system. With the remoting and automation of radar operations, there will be dramatic changes in the number and role of operators, as well as the displays required to support operators during missions. To design the new modernized system it is necessary to obtain a full understanding of how the current system operates and ensure that the allocation of tasks to humans and automation fully covers the entire operation. Traditional task analysis does not take into consideration plans for staffing reduction and addition of the new team “member” or agent, i.e., the automated system. To facilitate operator interaction with the modernized system, a new team architecture must be considered. CTA documents current operations, including a representation of mental activities, e.g., assessments, judgements, and decisions, in addition to the physical actions, e.g., executing system commands. A team CTA method is used to model the operations independent of the current operator roles and team structure. This approach captures the element of “team” by assuming a meta-operator and representing parallel tasking and data flow among tasks. This paper provides a description of the team CTA method and its application to large-scale, complex system design.

Post-pass partitioning of signal processing programs

Symmetric multiprocessor systems are increasingly common, not only as high-throughput servers, but as a vehicle for executing a single application in parallel in order to reduce its execution latency. This article presents Pedigree, a compilation tool that employs a new partitioning heuristic based on the program dependence graph (PDG). Pedigree creates overlapping, potentially interdependent threads, each executing on a subset of the SMP processors that matches the thread’s available parallelism. A unified framework is used to build threads from procedures, loop nests, loop iterations, and smaller constructs. Pedigree does not require any parallel language support; it is post-compilation tool that reads in object code. The SDIO Signal and Data Processing Benchmark Suite has been selected as an example of real-time, latency-sensitive code. Its coarse-grained data flow parallelism is naturally exploited by Pedigree to achieve speedups of 1.63×/2.13× (mean/max) and 1.71×/2.41× on two and four processors, respectively. There is roughly a 20% improvement over existing techniques that exploit only data parallelism. By exploiting the unidirectional flow of data for coarse-grained pipelining, the synchronization overhead is typically limited to less than 6% for synchronization latency of 100 cycles, and less than 2% for 10 cycles.

Development, Analysis, and Verification of a Parallel Hybrid Dataflow Computer Architectural Framework and Associated Load-Balancing Strategies and Algorithms via Parallel Simulation

This paper first presents the requirements for and the architectural framework of a type of real-time parallel dataflow computer architecture, specifically, a Hybrid Data/Command Driven Architecture (HDCA). Application of such an architecture is discussed. Goals, strategies, and algorithms for both static load-balancing and dynamic resource allocation on an HDCA system are presented. A parallel simulation of an HDCA system employing the presented static load-balancing and dynamic resource allocation algorithms is developed and presented. The parallel simulation was executed on a parallel computer system for increased fidelity. Analysis of simulation results for two example applications of the architecture allowed verification of design goals and correct performance of the HDCA architectural framework and associated static load-balancing and dynamic resource allocation strategies and algorithms.

Massively Parallel Dataflow Computer Dedicated to Real-Time Image Processing

This paper describes a massively parallel dataflow computer dedicated to real-time image processing. Two types of data-driven processing elements are embedded into a regular cubic mesh: 1024 low level custom dataflow processors (50Mops/s each) and 36

Implementation of a FFT radix 2 butterfly using serial RSFQ multiplier-adders

We have designed a Decimation-in-Time (DIT) radix 2 butterfly integrated circuit. This circuit will be used to implement the 32-point Fast Fourier Transform (FFT) in a parallel data flow architecture. The radix 2 butterfly circuit uses serial RSFQ math and consists of four single bit-wide serial multipliers and eight carry-save serial adders. The circuit with 16-bit word-length employs only 3400 junctions, occupies an area of 3.8/spl times/2.0 mm/sup 2/, and dissipates less than 1.1 mW power. The multiplier is implemented using the unique RSFQ bit-clock-pipelined schema. We have successfully tested a library of serial multiply-add elements: the 8-bit multiplier at 6.3 GHz and adders with dc bias margin /spl plusmn/20%. Finally, we have demonstrated full operation of the radix 2 butterfly chip with 5-bit word length.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Region analysis: a parallel elimination method for data flow analysis

Parallel data flow analysis methods offer the promise of calculating detailed semantic information about a program at compile-time more efficiently than sequential techniques. Previous work on parallel elimination methods (Zobel, 1990) has been hampered by the lack of control over interval size; this can prohibit effective parallel execution of these methods. To overcome this problem, we have designed the region analysis method, a new elimination method for data flow analysis. Region analysis emphasizes flow graph partitioning to enable better load balancing in a more effective parallel algorithm. We present the design of region analysis and the empirical results we have obtained that indicate: the prevalence of large intervals in flow graphs derived from real programs; and the performance improvement of region analysis over parallel Allen-Cocke interval analysis. Our implementation analyzed programs from the Perfect Benchmarks and netlib running on a Sequent Symmetry S81. >