General-purpose Accelerator Research Articles

Vision Transformer-based overlay processor for Edge Computing

Accelerating Visual Neural Networks in Edge Computing environments is crucial for processing image and video data. Visual Neural Networks, including Convolutional Neural Networks and Vision Transformers, are central to image recognition, video analysis, and object detection tasks. Deploying these networks on edge devices and accelerating them can significantly enhance data processing speed and efficiency. The large number of parameters, complex computational flows, and various structural variants of Transformer models present both opportunities and challenges. We propose Vis-TOP (Vision Transformer Overlay Processor), an overlay processor designed for all types of Vision Transformer models. Vis-TOP, distinct from coarse-grained general-purpose accelerators like GPUs and fine-grained custom designs, encapsulates Vision Transformer characteristics into a three-layer, two-level mapping structure, enabling flexible model switching without hardware architecture modifications. Concurrently, we designed a corresponding instruction bundle and hardware architecture within this mapping structure. We implemented the overlay processor design on the ZCU102 after quantizing the Swin Transformer model to 8-bit fixed points (fix_8). Experimentally, our throughput surpasses GPU implementation by 1.5 times. Our throughput per DSP is 2.2 to 11.7 times higher than that of existing Transformer-like accelerators. Overall, our approach satisfies real-time AI requirements in resource consumption and inference speed. Vis-TOP offers a cost-effective image processing solution for Edge Computing on reconfigurable devices, enhancing computational resource utilization, saving data transfer time and costs, and reducing latency.

A Generalized Block-Matrix Circuit for Closed-Loop Analog In-Memory Computing

Matrix-based computing is ubiquitous in an increasing number of present-day machine learning applications such as neural networks, regression and 5G communications. Conventional systems based on von-Neumann architecture are limited by the energy and latency bottleneck induced by the physical separation of the processing and memory units. Inmemory computing (IMC) is a novel paradigm where computation is performed directly within the memory, thus eliminating the need for constant data transfer. IMC has shown exceptional throughput and energy efficiency when coupled with crosspoint arrays of resistive memory devices in open-loop matrix-vector-multiplication and closed-loop inverse-matrix-vector multiplication (IMVM) accelerators. However, each application result in a different circuit topology, thus complicating the development of reconfigurable, general-purpose IMC systems. In this paper, we present a generalized closed-loop IMVM circuit capable of performing any linear matrix operation by proper memory remapping. We derive closed-form equations for the ideal input/ output transfer functions, static error, and dynamic behavior, introducing a novel continuous-time analytical model allowing for orders-of-magnitude simulation speedup with respect to SPICE-based solvers. The proposed circuit represents an ideal candidate for general-purpose accelerators of machine learning.

Intermediate Representations for Explicitly Parallel Programs

While compilers generally support parallel programming languages and APIs, their internal program representations are mostly designed from the sequential programs standpoint (exceptions include source-to-source parallel compilers, for instance). This makes the integration of compilation techniques dedicated to parallel programs more challenging. In addition, parallelism has various levels and different targets, each of them with specific characteristics and constraints. With the advent of multi-core processors and general purpose accelerators, parallel computing is now a common and pervasive consideration. Thus, software support to parallel programming activities is essential to make this technical transition more realistic and beneficial. The case of compilers is fundamental as they deal with (parallel) programs at a structural level, thus the need for intermediate representations. This article surveys and discusses attempts to provide intermediate representations for the proper support of explicitly parallel programs. We highlight the gap between available contributions and their concrete implementation in compilers and then exhibit possible future research directions.

Automatic FPGA-based Hardware Accelerator Design: A Case Study with Image Processing Applications

We present a case study of automatic FPGA-based hardware accelerator design using our proposed framework with the image processing domain. With the framework, the ultimate systems are optimized in both performance and energy consumption. Moreover, using the framework, designers can implement FPGA platforms without manually describing any hardware cores or the interconnect. The systems offer accelerations in execution time compared to traditional general purpose processors and accelerator systems designed manually. We use two applications in the image processing domain as experiments to report our work. Those are Canny edge detection and jpeg converter. The experiments are conducted in both embedded and high-performance computing platforms. Results show that we achieve overall speed-ups by up to 3.15´ and 2.87´ when compared to baseline systems in embedded and high-performance platforms, respectively. Our systems consume less energy than other FPGA-based systems by up to 66.5%.

Towards General-Purpose Acceleration: Finding Structure in Irregularity

Programmable hardware accelerators (e.g., vector processors, GPUs) have been extremely successful at targeting algorithms with regular control and memory patterns to achieve order-of-magnitude performance and energy efficiency improvements. However, they perform far under the peak on important irregular algorithms, like those from graph processing, database querying, genomics, advanced machine learning, and others. This work posits that the primary culprit is specific forms of irregular control flow and memory access. By capturing the problematic behavior at a domain-agnostic level, we propose an accelerator that is sufficiently general, matches domain-specific accelerator performance, and significantly outperforms traditional CPUs and GPUs.

All-Digital Energy-Constrained Controller for General-Purpose Accelerators and CPUs

Considering the energy-cap problem in battery-powered devices, dynamic voltage and frequency scaling, and power gating represent the de-facto state-of-the-art actuators. However, the limited margin available to reduce the operating voltage, the impossibility to massively integrate such actuators on-chip together with their actuation latency force a revision of such design methodologies. We present an all-digital architecture and a design methodology that can effectively manage the energy-cap problem for CPUs and accelerators. Two quality metrics are put forward to capture the performance loss and the energy budget violations. We employed a vector processor supporting four hardware threads as representative usecase. Results show an average performance loss and energy cap violations limited to 2.9% and 3.8%, respectively. Compared with solutions employing the dynamic frequency scaling actuator, our all-digital architecture improves the energy-cap violations by $3\times $ while maintaining a similar performance loss.

PIM-WEAVER: A High Energy-efficient, General-purpose Acceleration Architecture for String Operations in Big Data Processing

The ever-growing data volume in big data era drives designers to find more efficient processor architectures both in performance and energy consumption. Among various computing patterns in big data applications, string operations are common but important parts of data processing. However, due to the consideration of generality, current general-purpose CPUs are not efficient in both performance and energy consumption when processing simple and fixed computation patterns of discrete string operations. On the other hand, moving massive data from memory to computing units through NoC, Cache hierarchies, and other memory access data paths is time-consuming and especially energy consuming. Fortunately, emerging technologies, such as Hybrid Memory Cube (HMC), enable the processing-in-memory (PIM) functionality without transferring massive data to remote processing units. In this paper, we propose PIM-WEAVER, a high-efficiency novel acceleration architecture for string processing using PIM mechanism, which is the 3D integration technology that facilitates stacking logic and memory dies in a single package. The PIM-WEAVER is implemented using such technology by integrating string processing units into the real world HMC memory. In PIM-WEAVER, the general-purpose acceleration architecture for string operation is implemented within the memory cube, which can reduce the latency of data transfer and also save energy. We also propose a full-stack solution of programming interface and control mechanism of instruction level. Our comprehensive evaluations using typical string processing algorithms from big data applications show that the PIM-WEAVER gains an average speedup of 14.74x over high-performance Intel processor, and reduces the energy consumption by 82.1% on average, with tiny area overhead.

The Evaluation of DCNN on Vector-SIMD DSP

In current computer vision tasks, deep convolutional neural networks (DCNN) achieve state-of-the-art results. At present, the DCNN has been widely used in many usage scenarios. It is also important to develop DCNN on supercomputers in the age of artificial intelligence. As a general-purpose accelerator deployed in a supercomputer which ranks the fourth in the latest Top500 list, the Matrix2000 is a vector single instruction multiple data (vector-SIMD) digital signal processor (DSP). The research of DCNN implementation based on DSP is important and meaningful. Till now, there are few systematic studies published on DCNN implementation based on DSP. Qualcomm, CEVA, and Cadence have said that they have implemented DCNN on their own DSP. However, Qualcomm and CEVA have not published how they implemented DCNN based on their own DSP and Cadence has only implemented a convolution layer based on Vision P6. In this paper, we proposed a vectorization mapping method and a high efficient partition analysis model for implementing DCNN on vector-SIMD DSP. Based on the vectorization mapping method and analysis model, we implemented all layers of typical DCNN models on Matrix2000 and tested the computation and energy efficiency. The experiments demonstrate that the average computation efficiency of this paper based on Matrix2000 is 20~35% higher than GPU, 35~45% higher than Xeon Phi, about 8% higher than Vision P6 DSP, about 62~75% higher than an existing evaluation of DCNN based on Matrix2000, and the average energy efficiency is about 9~30% higher than GPU, and about 56% higher than the existing evaluation of DCNN based on Matrix2000. The results show that the vector-SIMD DSP with a suitable programming mapping method is also a suitable platform in the age of artificial intelligence.

Coherent Receiver DSP Implemented on a General-Purpose Server for a Full Software-Defined Access System

To accommodate the service paradigm shift driven by the now amazingly wide variety of services, telecom operators must overcome many challenges in redesigning their network infrastructure to attain system flexibility, scalability, and cost effectiveness. A key part of the solution is the recent advent of network virtualization based on commodity hardware. We propose a unique edge-server architecture that combines network virtualiza- tion with advanced digital signal processing (DSP) technology. Our architecture has the server's central processing unit running application layer functions and a generalpurpose accelerator running dissimilar physical layer functions, including advanced DSP. To elucidate the power budget capability of our coherent detection scheme, a hardware experiment demonstrates the optical power budget of 45.2 dB for quadrature phase-shift keying signals of 5 Gbit/s bit rate.

Auto-Tuning CNNs for Coarse-Grained Reconfigurable Array-Based Accelerators

As more and more deep learning tasks are pushed to mobile devices, accelerators for running these networks efficiently gain in importance. We show a that an existing class of general purpose accelerators, modulo-scheduled coarse-grained reconfigurable array (CGRA) processors typically used to accelerate multimedia workloads, can be a viable alternative to dedicated deep neural network processing hardware. To this end, an auto-tuning compiler is presented that maps convolutional neural networks (CNNs) efficiently on such architectures. The auto-tuner analyzes the structure of the CNN and the features of the CGRA, then explores the large optimization space to generate code that allows for an efficient mapping of the network. Evaluated with various CNNs, the auto-tuned code achieves an 11-fold speedup over the initial mapping. Comparing the energy per interference, the CGRA outperforms other general-purpose accelerators and an ARMv8 processor by a significant margin.

Numerical algorithms for scatter-to-attenuation reconstruction in PET: empirical comparison of convergence, acceleration, and the effect of subsets.

MLGA was combined with the Armijo step size rule; and accelerated using conjugate gradients, Nesterov's momentum method, and data subsets of different sizes. In 2BP, we varied the subset size, an important determinant of convergence speed and computational burden. We used three sets of simulation data to evaluate the impact of a spatial scale factor. The Armijo step size allowed 10-fold increased step sizes compared to native MLGA. Conjugate gradients and Nesterov momentum lead to slightly faster, yet non-uniform convergence; improvements were mostly confined to later iterations, possibly due to the non-linearity of the problem. MLGA with data subsets achieved faster, uniform, and predictable convergence, with a speed-up factor equivalent to the number of subsets and no increase in computational burden. By contrast, 2BP computational burden increased linearly with the number of subsets due to repeated evaluation of the objective function, and convergence was limited to the case of many (and therefore small) subsets, which resulted in high computational burden. Possibilities of improving 2BP appear limited. While general-purpose acceleration methods appear insufficient for MLGA, results suggest that data subsets are a promising way of improving MLGA performance.

Large-scale fast Fourier transform on a heterogeneous multi-core system

As interest in hybrid computing systems increases, people are eager to find new ways to exploit the unique and efficient computational power of the heterogeneous multi-core systems. Although there has been much interest in implementing high-performance fast Fourier transform (FFT) libraries on this kind of system, most existing libraries focus on small-scale FFTs whose data can fit in the local storage of a single accelerator. Real-world FFT applications often require much larger scale FFTs, but it is extremely challenging for heterogeneous multi-core system with distributed architectures to make efficient large FFT implementations. In this paper, we introduce the first known FFT library for the heterogeneous multi-core system with distributed architecture that can solve one-dimensional FFTs larger than what fits in a single accelerator. Our implementation achieves 67% performance improvement of FFTW 3.2.2 (Fastest Fourier Transform in the West) and sustains over 36 single precision FFT GFLOPs ‘end-to-end’ Achieving such high performance requires novel schemes for large-scale FFT factorization, data permutation and all-to-all exchanges, and buffer designs to maximize use of the local storage while minimizing communication overhead. One important finding in this paper is that large-scale FFT on this kind of architecture behaves as data transfer bound which is quite different from other architectures. A significant contribution of this paper is that for each major component of our algorithm, we explore many possible design options and present quantitative performance comparisons. This provides value beyond specific architecture, as it illustrates the fundamental features associated with different communication paradigms and mechanisms for the heterogeneous multi-core system. Today’s computer systems are increasingly being designed to include general purpose accelerators. The techniques in this paper can also be applied to these architectures especially when they have limited local storage or steep cache hierarchies. We also provide insights on applying techniques in this paper to similar architectures.

Heterogeneous Multicore SoC With SiP for Secure Multimedia Applications

A heterogeneous multicore system-on-chip (SoC) has been developed for high-definition (HD) multimedia applications that require secure DRM (digital rights management). The SoC integrates three types of processors: two specific-purpose accelerators for cipher and high-resolution video decoding; one general-purpose accelerator (MX); and three CPUs. This is how our SoC achieves high performance and low power consumption with hardware customized for video processing applications that process a large amount of data. To achieve secure data control, hardware memory management and software system virtualization are adopted. The security of the system is the result of the cooperation between the hardware and software on the system. Furthermore, a highly tamper-resistant system is provided on our SiP (System in a package), through DDR2 SDRAMs and a flash memory that contain confidential information in one package. This secure multimedia processor provides a solution to protect contents and to safely deliver secure sensitive information when processing billing transactions that involve digital content delivery. The SoC was implemented in a 90 nm generic CMOS technology.

Vector Processing as a Soft Processor Accelerator

Current FPGA soft processor systems use dedicated hardware modules or accelerators to speed up data-parallel applications. This work explores an alternative approach of using a soft vector processor as a general-purpose accelerator. The approach has the benefits of a purely software-oriented development model, a fixed ISA allowing parallel software and hardware development, a single accelerator that can accelerate multiple applications, and scalable performance from the same source code. With no hardware design experience needed, a software programmer can make area-versus-performance trade-offs by scaling the number of functional units and register file bandwidth with a single parameter. A soft vector processor can be further customized by a number of secondary parameters to add or remove features for a specific application to optimize resource utilization. This article introduces VIPERS, a soft vector processor architecture that maps efficiently into an FPGA and provides a scalable amount of performance for a reasonable amount of area. Compared to a Nios II/s processor, instances of VIPERS with 32 processing lanes achieve up to 44× speedup using up to 26× the area.

Radiocarbon AMS at IOP: System improvements and dating of groundwater from Bhadrak district, Orissa

The radiocarbon AMS facility at Institute of Physics, Bhubaneswar is based on a 3MV tandem accelerator. In this paper, we present the recent improvements to the AMS system at this general purpose accelerator facility. We report an effective method of overcoming terminal voltage instability often met with in switching between AMS and other modes of operation of the accelerator. We report the radiocarbon measurements made on connate groundwater samples from the Bhadrak district of Orissa state (eastern India) to identify the regions that require artificial recharge.

A general purpose accelerator for digital system design

With the growing acceptance of simulation tools for the analysis and verification of digital circuits, and the ever increasing complexity of digital system designs involving multiple ASICs and complex commercial VLSI components, designers are demanding increased performance from the design tools. This paper outlines the problems associated with accelerating design verification tools focused for system-level design. The architecture of a commercially available general purpose hardware accelerator is then described to see how these problems can be addressed.

A Supercomputer Workstation for VLSI CAD

This article describes a VLSI CAD workstation with a massively parallel computer the connection machine, as hardware accelerator. The connection machine offers workstation users general-purpose acceleration capabilities and high interactivity. Workstation software includes a novel CAD-system kernel and tools operating on the connection machine. The system kernel, designed to permit efficient interfaces to existing tools and tool environments, also includes more advanced design tools (procedural tools, for example) providing a suitable basis for knowledge-based tool development.