Hybrid Dataflow Research Articles

Dadu-SV: Accelerate Stereo Vision Processing on NPU

Binocular vision and neural networks (CNNs) are widely seen in modern intelligent vision processing systems, such as robotics, autonomous vehicles, and AR gadgets. However, both the classic semiglobal matching (SGM) and deep CNNs entail substantial computing resource to reach the performance goal. Traditional embedded CPU/graphic processor unit (GPU) cannot simultaneously meet the processing speed and energy requirement, while the specialized circuits dedicated to SGM and CNN processing, respectively, will take considerable hardware and development costs. However, as the popularity of deep learning, neural processing units (NPUs) become prevalent in many embedded and edge devices, which possess high throughput computing power to deal with the matrix operations involved by neural networks. In this work, we attempt to take advantage of the neural processing architectures integrated in SoC chips to accelerate the SGM process, so that the hardware resources will be better utilized instead of investing more resources to create specialized SGM components. Thereby, this letter first deploys SGM on NPU by converting the incompatible operations into the neural-computing flow, and a configurable neural processing element is proposed to flexibly support various vector operation sequences. Then, a hybrid dataflow scheduler and the corresponding hardware modification are introduced to accelerate the cost processing, improving hardware utilization and on-chip memory footprint and access. Our solution runs at 45 fps for an image size of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$640\times 480$ </tex-math></inline-formula> , with 128 disparity levels. The speed-energy efficiency is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$52\times $ </tex-math></inline-formula> better than the GPU (Jetson TX1) solution with negligible additional hardware overhead and accuracy loss.

A Peak Prediction Method for Subflow in Hybrid Data Flow

Subflow prediction is required in resource active elastic scaling, but the existing single flow prediction methods cannot accurately predict the peak variation of subflow in hybrid data flow. These do not consider the correlation between subflows. The difficulty is that it is hard to calculate the correlation between different data flows in hybrid data flow. In order to solve this problem, this paper proposes a new method DCCSPP (subflow peak prediction of hybrid data flow based on delay correlation coefficients) to predict the peak value of hybrid data flow. Firstly, we establish a delay correlation coefficient model based on the sliding time window to determine the delay time and delay correlation coefficient. Next, based on the model, a hybrid data flow subflow peak prediction model and algorithm are established to achieve accurate peak prediction of subflow. Experiments show that our prediction model has achieved better results. Compared with LSTM, our method has decreased the MAE about 18.36% and RMSE 13.50%. Compared with linear regression, MAE and RMSE are decreased by 27.12% and 25.58%, respectively.

Copernicus, a hybrid dataflow and peer-to-peer scientific computing platform for efficient large-scale ensemble sampling

Compute-intensive applications have gradually changed focus from massively parallel supercomputers to capacity as a resource obtained on-demand. This is particularly true for the large-scale adoption of cloud computing and MapReduce in industry, while it has been difficult for traditional high-performance computing (HPC) usage in scientific and engineering computing to exploit this type of resources. However, with the strong trend of increasing parallelism rather than faster processors, a growing number of applications target parallelism already on the algorithm level with loosely coupled approaches based on sampling and ensembles. While these cannot trivially be formulated as MapReduce, they are highly amenable to throughput computing. There are many general and powerful frameworks, but in particular for sampling-based algorithms in scientific computing there are some clear advantages from having a platform and scheduler that are highly aware of the underlying physical problem. Here, we present how these challenges are addressed with combinations of dataflow programming, peer-to-peer techniques and peer-to-peer networks in the Copernicus platform. This allows automation of sampling-focused workflows, task generation, dependency tracking, and not least distributing these to a diverse set of compute resources ranging from supercomputers to clouds and distributed computing (across firewalls and fragile networks). Workflows are defined from modules using existing programs, which makes them reusable without programming requirements. The system achieves resiliency by handling node failures transparently with minimal loss of computing time due to checkpointing, and a single server can manage hundreds of thousands of cores e.g. for computational chemistry applications.

DaSH: A benchmark suite for hybrid dataflow and shared memory programming models

Abstract The current trend in development of parallel programming models is to combine different well established models into a single programming model in order to support efficient implementation of a wide range of real world applications. The dataflow model has particularly managed to recapture the interest of the research community due to its ability to express parallelism efficiently. Thus, a number of recently proposed hybrid parallel programming models combine dataflow and traditional shared memory models. Their findings have influenced the introduction of task dependency in the OpenMP 4.0 standard. This article presents DaSH – the first comprehensive benchmark suite for hybrid dataflow and shared memory programming models. DaSH features 11 benchmarks, each representing one of the Berkeley dwarfs that capture patterns of communication and computation common to a wide range of emerging applications. DaSH also includes sequential and shared-memory implementations based on OpenMP and Intel TBB to facilitate easy comparison between hybrid dataflow implementations and traditional shared memory implementations based on work-sharing and/or tasks. Finally, we use DaSH to evaluate three different hybrid dataflow models, identify their advantages and shortcomings, and motivate further research on their characteristics.

Kleene Algebra and Bytecode Verification

Most standard approaches to the static analysis of programs, such as the popular worklist method, are first-order methods that inductively annotate program points with abstract values. In [6] we introduced a second-order approach based on Kleene algebra. In this approach, the primary objects of interest are not the abstract data values, but the transfer functions that manipulate them. These elements form a left-handed Kleene algebra. The dataflow labeling is not achieved by inductively labeling the program with abstract values, but rather by computing the star (Kleene closure) of a matrix of transfer functions. In this paper we show how this general framework applies to the problem of Java bytecode verification. We show how to specify transfer functions arising in Java bytecode verification in such a way that the Kleene algebra operations (join, composition, star) can be computed efficiently. We also give a hybrid dataflow analysis algorithm that computes the closure of a matrix on a cutset of the control flow graph, thereby avoiding the recalculation of dataflow information when there are cycles in the graph. This method could potentially improve the performance over the standard worklist algorithm when a small cutset can be found.

Prototyping of Hybrid Systems – From HyCharts to Hybrid Data-Flow Graphs

In this paper, a translation of the visual description technique HyCharts to Hybrid Data-Flow Graphs (HDFG) is given. While HyCharts combine a data-flow and a control-flow oriented formalism for the specification of the architecture and the behavior of hybrid systems, HDFG allow the efficient and homogeneous internal representation of hybrid systems in computers and their automatic manipulation. HDFG represent a system as a data-flow network built from a set of fundamental functions.The translation permits to combine the advantages of the different description techniques: The use of HyCharts for specification supports the abstract and formal interactive specification of hybrid systems, while HDFG permit the tool based optimization of hybrid systems and the synthesis of mixed-signal prototypes.

Development, Analysis, and Verification of a Parallel Hybrid Dataflow Computer Architectural Framework and Associated Load-Balancing Strategies and Algorithms via Parallel Simulation

This paper first presents the requirements for and the architectural framework of a type of real-time parallel dataflow computer architecture, specifically, a Hybrid Data/Command Driven Architecture (HDCA). Application of such an architecture is discussed. Goals, strategies, and algorithms for both static load-balancing and dynamic resource allocation on an HDCA system are presented. A parallel simulation of an HDCA system employing the presented static load-balancing and dynamic resource allocation algorithms is developed and presented. The parallel simulation was executed on a parallel computer system for increased fidelity. Analysis of simulation results for two example applications of the architecture allowed verification of design goals and correct performance of the HDCA architectural framework and associated static load-balancing and dynamic resource allocation strategies and algorithms.

Modeling, design, and performance analysis of a parallel hybrid data/command driven architecture system and its scalable dynamic load balancing circuit

Addresses a proposed parallel hybrid dataflow architecture, a scalable dynamic load balancing circuit for the proposed architecture, and performance analysis, first, of the load balancing circuit and, second, the architecture using the load balancing circuit. The contributions and focus of this paper are: (1) that it first describes the requirements for and the framework of a parallel, medium to coarse grain, hybrid token controlled dataflow architecture. This paper only deals with real-time applications of the architecture. (2) The paper next describes the dynamic load balancing strategy for the hybrid dataflow architecture and a resulting mathematical model of the load balancing function required by the load balancing strategy. (3) The organization, design, and implementation of a basic digital circuit suitable for VLSI implementation which implements the mathematical model of the load balancing function required by the architecture is next presented. This circuit implements a control token mapping function and is therefore called a "token mapper". (4) It is next shown that the basic dynamic load balancing circuit (token mapper) design is scalable therefore allowing the hybrid dataflow architecture to be scalable. (5) The performance of the dynamic load balancing circuit is then analyzed at both the circuit and architectural systems level. A parallel simulation of the proposed parallel hybrid dataflow architecture employing its presented dynamic load balancing circuit was developed for two example applications and used for architectural system level performance analysis. Analysis of simulation results verified correct operation of the proposed hybrid dataflow system architecture and its dynamic load balancing circuit.

An Efficient Hybrid Dataflow Architecture Model

In this paper, we present an efficient hybrid architecture model which employee (1) a conventional architecture technique to achieve fast pipelined instruction execution, while exploiting fine-grain parallelism by data-driven instruction scheduling; (2) an efficient mechanism which supports concurrent operation of multiple instruction threads on the hybrid architecture model; (3) a compiling paradigm for dataflow software pipelining which efficiently exploits loop parallelism through limited balancing. We establish a set of basic results which show that the fine-grain parallelism in a loop exposed through limited balancing can be fully exploited by a simple greedy runtime data-driven scheduling scheme, achieving both time and space efficiency simultaneously. Simulation results are briefly discussed. This architecture model is utilized as a research vehicle to study various architecture issues for hybrid dataflow computers, as well as compiling techniques for such machines.

MIMD COMPUTERS FOR SCIENTIFIC APPLICATIONS

The purpose of this talk is twofold: to give an overview of the major parallel processor projects at IBM T.J. Watson research and then to discuss in some detail the VICTOR [1] message-passing parallel machine. Finally, concepts of a possible massively-parallel machine for very high performance are briefly discussed. In the first part of the talk the RP3 [2], the ACE [3], the GFII [4], the M/370, the Hybrid Dataflow [5], and the VLIW [6] projects will be discussed at a level sufficient to explain the key features of each of these projects, but without going into technical details. In summary, the RP3 is a 64-node shared memory machine, the ACE a 8-node shared bus workstation, the GF11 a switch-based SIMD machine with up to 576 nodes, the M/370 a project to study commercial applications of message-passing machines. The Hybrid Data Flow and the VLIW projects intend to explore very fine grained parallelism. The VICTOR project is a family of transputer based machines which were designed and build at T.J. Watson. It includes V32 and V256 with 32 and 256 nodes, respectively, and a set of 16-node workstations. All of these are presently operational. All machines except V32 are T800 based, with 4 MB of memory/node. V256 incorporates a distributed 16-node fileserver with 10 GB capacity. Special hardware supports multiple-users via spatial partitioning and non-intrusive processor monitoring. The operating environment is based on a mixture of vendor compiler and loader technology and a runtime system developed in the VICTOR group. In particular, this includes a set of routing routines and support for the filesystem. Languages used are OCCAM, C, Pascal and Fortran 77. Presently, the Trollius [7] operating system which has been developed at Cornell University is being ported onto VICTOR. Applications programs which have been written for VICTOR include presently fractals, ray-tracing, a nuclear physics Monte Carlo code, a computer pipeline model and a neural network code. Under development presently are two codes for VLSI design, dealing with parallel fault simulation and with circuit simulation. The latter one uses a parallel version of waveform relaxation techniques and is expected to be able to handle circuits with up to 106 transistors. Other applications being developed are graphical process monitors, database research, 3D-seismic code and multi-robot simulation. Work in the Mathematical Sciences department is focussed on developing a parallel language with emphasis on dynamic process scheduling. In the last part of the talk issues of scaling message-passing architectures into the Teraflop performance range are discussed.

A high-speed memory organization for hybrid dataflow / von Neumann computing

Multi-threaded node architectures have been proposed as a promising means to tolerate latency of memory accesses in a multiprocessor system. In this paper, we propose a novel organization of high-speed memories, known as the register-cache, for a multi-threaded architecture. As the term suggests, it is organized both as a register file and a cache. Viewed from the execution unit, its contents are addressable as ordinary CPU registers using relatively short addresses. From the main memory perspective, it is content addressable, i.e. its contents are tagged just as in conventional caches. In this register-cache organization, a number of registers are grouped into a block of registers where a register in a block is accessed using an offset from the address of the block, an offset value which is embedded in the compiler generated code. The binding of register block locations to register-cache line addresses is adaptively performed at runtime, thus resulting in a dynamically allocated register file. In this execution model, a program is compiled into a number of instruction threads called super-actors. A super-actor becomes ready for execution only when its input data are physically residing in the register-cache and space is reserved in the register-cache to store its result. Therefore, the execution unit will never stall or ‘freeze’ when accessing instruction or data. Another advantage is that since registers are dynamically assigned at runtime, register allocation difficulties at compile-time, e.g. allocating registers for subscripted variables of large arrays, can be avoided. Architectural support for overlapping executions of super-actors and main memory operations are provided so that the available concurrency in the underlying machine can be better utilized. The preliminary simulation results seem to be very encouraging: with software pipelined loops, a register-cache of moderate size can keep the execution unit usefully busy.

Thread-based programming for the EM-4 hybrid dataflow machine

In this paper, we present a thread-based programming model for the EM-4 hybrid dataflow machine, where parallelism and synchronization among threads of sequential execution are described explicitly by the programmer. Although EM-4 was originally designed as a dataflow machine, we demonstrate that it provides effective architectural support for a variety of programming styles, including message passing and distributed data sharing in imperative languages. Our approach allows the programmer to control the parallelism and maintain data locality explicitly to achieve high performance. EM-4 can be thought of as a multi-threaded architecture that can exploit both von Neumann and dataflow compiling technology. Thread-based programming provides the first step to explore better programming/compiling technology for a hybrid dataflow machine as well as EM-4.