Adaptive Data Flow Research Articles

ADS-CNN: Adaptive Dataflow Scheduling for lightweight CNN accelerator on FPGAs

Lightweight convolutional neural networks (CNNs) enable lower inference latency and data traffic, facilitating deployment on resource-constrained edge devices such as field-programmable gate arrays (FPGAs). However, CNNs inference requires access to off-chip synchronous dynamic random-access memory (SDRAM), which significantly degrades inference speed and system power efficiency. In this paper, we propose an adaptive dataflow scheduling method for lightweight CNN accelerator on FPGAs named ADS-CNN. The key idea of ADS-CNN is to efficiently utilize on-chip resources and reduce the amount of SDRAM access. To achieve the reuse of logical resources, we design a time division multiplexing calculation engine to be integrated in ADS-CNN. We implement a configurable module for the convolution controller to adapt to the data reuse of different convolution layers, thus reducing the off-chip access. Furthermore, we exploit on-chip memory blocks as buffers based on the configuration of different layers in lightweight CNNs. On the resource-constrained Intel CycloneV SoC 5CSEBA6 FPGA platform, we evaluated six common lightweight CNN models to demonstrate the performance advantages of ADS-CNN. The evaluation results indicate that, compared with accelerators that use traditional tiling strategy dataflow, our ADS-CNN can achieve up to 1.29× speedup with the overall dataflow scale compression of 23.7%.

Algorithm/Hardware Co-Optimization for Sparsity-Aware SpMM Acceleration of GNNs

In recent years, graph neural networks (GNNs) have achieved impressive performance in various application fields by extracting information from graph-structured data. It contains extensive feature aggregation operations and has become a performance bottleneck, which can be abstracted as a specialized Sparse-Dense Matrix Multiplication (SpMM) operation. Previous works have leveraged the inner product or outer product to accelerate the feature aggregation process. However, inefficient execution leads to extremely unbalanced workloads and extensive intermediate data, hampering the performance of previous processors. So in this paper, we demonstrate an algorithm/hardware co-optimization chance to enhance SpMM acceleration for GNNs. First, the algorithm part develops a dataflow-efficient SpMM algorithm that integrates three optimization methods to mitigate computation and memory access inefficiencies. Specifically, 1) the proposed equal-value partition method achieves fine-grained data partition and enables load balancing during data movement. 2) After observing the vertex aggregation phenomenon, a vertex-clustering optimization method is presented to enable significant data locality. 3) The adaptive dataflow based on Gustavson’s algorithm is further implemented to enable the efficient distribution of sparse elements and improves computing resource utilization. Then, the hardware part features the proposed SpMM algorithm and customizes SDMA, a flexible and efficient accelerator to boost SpMM acceleration, which follows the adaptive dataflow to eliminate sparsity and explore the regular parallelism dimension. Finally, we prototype SDMA on the Xilinx Alveo U280 FPGA accelerator card. The results demonstrate that SDMA achieves 5.68× 14.68× energy efficiency over the previous GPU implementations deployed on the Nvidia GTX 1080Ti and 1.32× higher throughput over the state-of-the-art FPGA prototype.

IMGA: Efficient In-Memory Graph Convolution Network Aggregation With Data Flow Optimizations

Aggregating features from neighbor vertices is a fundamental operation in Graph Convolution Network (GCN). However, the sparsity in graph data creates poor spatial and temporal locality, causing dynamic and irregular memory access patterns and limiting the performance of aggregation on the Von Neumann architecture. The emerging processing-in-memory (PIM) architecture is based on emerging non-volatile memory (NVM), like Spin-orbit torque Magnetic RAM (SOT-MRAM), and demonstrates promising prospects in alleviating the Von Neumann bottleneck. However, the limited memory capacity of PIM medium still incurs non-negligible data movements between PIM architecture and external memory. To solve this challenge, we propose a SOT-MRAM based in-memory computing architecture, called IMGA, for efficient in-situ graph aggregation. Specifically, we design adaptive data flow management strategies that reuse vertex data in MRAM when processing graphs of different scales and adopt edge data as the control signal source to utilize the graph’s structural information. A reordering optimization strategy leveraging hardware-software co-design principle is proposed to further reduce the costly data movement. Experimental results demonstrate that IMGA achieves an average 2523x and 21x speedup, and 1.03E+6 and 1.04E+3 energy efficiency compared with CPU and GPU, respectively.

A real-time and high-performance MobileNet accelerator based on adaptive dataflow scheduling for image classification

A real-time and high-performance MobileNet accelerator based on adaptive dataflow scheduling for image classification

A Low-Latency Framework With Algorithm-Hardware Co-Optimization for 3-D Point Cloud

As an important type of 3D representation, the point cloud is widely used in many applications, such as autonomous driving, AR/VR, and intelligent robots, which require real-time interactions with humans. However, the sparsity of 3D point cloud data leads to severe computational inefficiency when being processed by 2D data processors, posing a huge challenge for hardware acceleration. In this paper, we aim at solving the inefficiency problem by algorithm-hardware co-optimization. Firstly, a lightweight network, named LPN, is proposed for point cloud data classification, which is 30× smaller than pointnet and still has comparable accuracy. Secondly, a reconfigurable computing core, named RCC, together with an adaptive dataflow, is developed to support different layers of the LPN. Specifically, to accelerate memory-intensive layers, a partially-parallel computing scheme is introduced to minimize the on-chip memory requirements and DRAM accesses. Finally, based on the above innovations, a low-latency accelerator is proposed to realize real-time computation for the point cloud, which is implemented on the Xilinx Kintex UltraScale KCU150 FPGA board. Experimental results show that it achieves 1.5× throughput improvement compared with the state-of-the-art works, and 35× speedup over Intel Xeon Gold 6148 CPU, demonstrating the superiority of the proposed method. The code of LPN is available from https://github.com/snowsil/LPN-model-for-3D-classification.git.

Trainer: An Energy-Efficient Edge-Device Training Processor Supporting Dynamic Weight Pruning

Transfer learning, which transfers knowledge from source datasets to target datasets, is practical for adaptive deep neural network (DNN) applications. When considering user privacy and communication bandwidth issues, edge devices’ training is essential for transfer learning. Nevertheless, training requires repeating feedforward (FF), backpropagation (BP), and weight gradient (WG) millions of times, introducing prohibitive computation for edge devices. A promising method to reduce training computation is sparse DNN training (SDT), which dynamically prunes weights during training iterations and performs FF, BP, and WG only with unpruned weights. However, SDT suffers implicit redundancy and reuse imbalance for convolution layers. Besides, it turns bottlenecks into batch normalization (BN) layers. Therefore, it is challenging to achieve energy-efficient SDT computing. This article proposes a processor, Trainer, solving the above challenges with three features. First, a speculation mechanism removes implicit redundant operations, which have nonzeros’ input, weight, or output, but are ineffective for training. Second, a dynamic sparsity adaptive dataflow tackles the reuse imbalance, improving energy efficiency (EE) for dynamic sparse convolution in SDT. Third, a computational dependence decoupled BN unit eliminates BN’s repeated data access to reduce training energy and time. Trainer is fabricated in 28-nm CMOS technology and occupies 20.96 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> of area. It achieves a peak EE of 173.28TFLOPS/W@FP16 (276.55TFLOPS/W@FP8) for a 90% activation sparsity and 90% weight sparsity. The sparsity to EE conversion ratio is 80.9, outperforming the previous work by 1.55 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> . When training a ResNet18 model with SDT, Trainer reduces energy by 2.23 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> and time by 1.76 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> than the state-of-the-art sparse training processor.

Concurrent agent‐based evolutionary computations as adaptive dataflows

SummaryThis paper introduces a new formal description of the execution model for agent‐based computing systems in the form of an adaptive dataflow decoupled from the domain‐specific semantics of the computation. We show that the execution models studied in previous work can be unified in this common model. The parameters of the model such as queuing policies and granularity of the data in the flow are analyzed. Several queueing alternatives are benchmarked to demonstrate how they affect the efficiency of the computation. Using the example of a multi‐agent evolutionary optimisation problem solver, the new approach is shown to outperform the classic one. This proposed model is well suited to functional languages and can be easily mapped onto different classes of hardware – from simple single‐core computers to distributed environments.

A Reactive and Adaptive Data Flow Model for Network-of-System Specification

With embedded systems being increasingly networked, appropriate models of computation and communication are needed for specification of such networks-of-systems. Traditional dataflow models have shown their usefulness in analyzing isolated systems. However, these models cannot express the inherent requirements of connected applications, such as dynamic behavior associated with network losses and reactivity to external events. This letter proposes a reactive and adaptive data flow (RADF) model that introduces a notion of empty tokens to expose network losses and provide adaptivity at the application level while maintaining overall determinism. Empty tokens, combined with expanded actor semantics, are also used to model reactivity to sporadic external events. We formally define RADF semantics and show efficient methods for analyzing RADF graphs in terms of their worst-case throughput and latency.

Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks

Eyeriss is a dedicated accelerator for deep neural networks (DNNs). It features a spatial architecture that supports an adaptive dataflow, called Row-Stationary (RS), which optimizes data movement in a multi-level storage hierarchy according to the shape and size of the DNN model. Unlike the previous work that commonly applies one-size-fits-all dataflows regardless of the data structure of the DNN model, we have shown quantitatively that much better results could be achieved with the highly adaptive RS dataflow. This analysis is made possible thanks to our framework that can systematically analyze and evaluate the system energy efficiency of different dataflows working for different shapes and sizes of DNNs. We observe that DRAM alone does not dictate energy efficiency, and optimizing the energy consumption for only a certain data type does not lead to the best overall system energy efficiency. For AlexNet, the RS dataflow is 1.4 to 2.5 times more energy efficient than existing dataflows in the convolutional layers, and at least 1.3 times more energy efficient in the fully-connected layers for batch size of at least 16.

Dataflow Graph Partitioning for Area-Efficient High-Level Synthesis with Systems Perspective

Area efficiency in datapath synthesis is a widely accepted goal of high-level synthesis. Applications represented by their dataflow graphs are synthesized using resource sharing principles to reduce the area. However, existing resource sharing algorithms focus on absolute area reduction and maximal resource sharing. This kind of a design approach leads to constraints on how often, in terms of number of clock cycles, a new set of input data can be fed to an application. It also leads to very large multiplexers in case of very big dataflow graphs with hundreds of nodes. An adaptive dataflow graph partitioning algorithm is proposed that partitions a graph taking into account a user-defined constraint on how often a new set of input data (generally referred to as data initiation interval) is available. At the same time, a resource sharing algorithm is applied to such partitions in order to reduce area. Multiple design points are generated for a given dataflow graph with different area and time measures to enable a designer to make decisions. We demonstrate our graph partitioning algorithm using synthetically generated large dataflow graphs and on some benchmark applications.

Scalable and adaptive online joins

Scalable join processing in a parallel shared-nothing environment requires a partitioning policy that evenly distributes the processing load while minimizing the size of state maintained and number of messages communicated. Previous research proposes static partitioning schemes that require statistics beforehand. In an online or streaming environment in which no statistics about the workload are known, traditional static approaches perform poorly. This paper presents a novel parallel online dataflow join operator that supports arbitrary join predicates. The proposed operator continuously adjusts itself to the data dynamics through adaptive dataflow routing and state repartitioning. The operator is resilient to data skew, maintains high throughput rates, avoids blocking behavior during state repartitioning, takes an eventual consistency approach for maintaining its local state, and behaves strongly consistently as a black-box dataflow operator. We prove that the operator ensures a constant competitive ratio 3:75 in data distribution optimality and that the cost of processing an input tuple is amortized constant, taking into account adaptivity costs. Our evaluation demonstrates that our operator outperforms the state-of-the-art static partitioning schemes in resource utilization, throughput, and execution time.

Force-reflecting teleoperation over the internet: the JBIT project

An ever-growing number of Internet-connected devices are now accessible to a multitude of users. Being an ubiquitous communication means, the Internet could enable any user to reach and command any device connected to the network. This paper reports the successful application of real-time closed-loop control over the Internet in the Java-Based Interface for Telerobotics (JBIT) system, in which the Internet users can access and command a two-degrees-of-freedom robot in real time, receiving both visual and force feedback. When closed-loop control of a remote system comes into play, careful evaluation of the performance and limits of the communication system in use is mandatory. The reported analysis shows that the main Internet limits are unknown available throughput, variable delay, and loss of some data packets, in particular when the network is congested. Once the limits of the communication system are known, it is shown that it is possible to use the Internet for the remote closed-loop control of a slave robot, provided that suitable strategies to guarantee operability and safety of the controlled system are implemented. Such strategies are concerned with on-line identification of the connection characteristics, adaptation of the data rate to the available throughput, recovery of the missing packet, and handling of the variable delay. Moreover, visual feedback dataflow shares the same Internet connection with the real-time control data. To prevent congestion, new tools for adaptive dataflow management have been developed. Finally, an improved coordinating force control scheme, aimed at enhancing the transparency of the teleoperator, is presented.