Dedicated Hardware Architecture Research Articles

Efficient FPGA implementation for sound source separation using direction-informed multichannel non-negative matrix factorization

Sound source separation (SSS) is a fundamental problem in audio signal processing, aiming to recover individual audio sources from a given mixture. A promising approach is multichannel non-negative matrix factorization (MNMF), which employs a Gaussian probabilistic model encoding both magnitude correlations and phase differences between channels through spatial covariance matrices (SCM). In this work, we present a dedicated hardware architecture implemented on field programmable gate arrays (FPGAs) for efficient SSS using MNMF-based techniques. A novel decorrelation constraint is presented to facilitate the factorization of the SCM signal model, tailored to the challenges of multichannel source separation. The performance of this FPGA-based approach is comprehensively evaluated, taking advantage of the flexibility and computational capabilities of FPGAs to create an efficient real-time source separation framework. Our experimental results demonstrate consistent, high-quality results in terms of sound separation.

Low-Power Inverse Multiple Transform Hardware Design for 8K@60fps Real-Time VVC Decoding

The Versatile Video Coding (VVC) is a new standard released in 2020 and is considered the state-of-the-art technology in video coding. It introduces several innovative tools to achieve high coding efficiency, but, on the other side, requires a significantly higher computational effort. One of the innovations of the standard is the Multiple Transform Selection (MTS), which plays a crucial role in improving coding efficiency. MTS supports square and rectangular transform blocks and includes three transform types (DCT-II, DCT-VIII, and DST-VII) that can be combined in horizontal and vertical directions. As a result, VVC codecs must support a wide range of transform types and formats, making the development of hardware solutions essential to enable real-time processing. This work presents a dedicated hardware architecture for all inverse transforms in the VVC decoder. The proposed architecture performs one-dimensional transforms for all sizes supported in the VVC standard, allowing for the combination into 2D square and rectangular transforms, ranging from 4×4 to 64×64 blocks for IDCT-II and from 4×4 to 32×32 for IDCT-VIII and IDST-VII. The proposed architecture can process videos of up to UHD 8K@60fps resolution in real time, with a power consumption of 778 mW.

Efficient Dedicated Hardware Design System for the VVC Low-Frequency Non-Separable Transform

This paper proposes a dedicated hardware architecture for the Low-Frequency Non-Separable Transform (LFNST) of the Versatile Video Coding (H.266/VVC) standard. The VVC defines two stages of transformation, where the first stage uses traditional transform types (e.g. DCT-II, DCT-VII and DST-VII), while the secondary transform stage applies the LFNST. The LFNST is used to transform the coefficients that were transformed by the DCT-II in the primary transform, but only those from the residues that came from the intra prediction. The developed LFNST system design exploits the Clock Crossing Domain technique to extract the best relation between performance and area/power. Consequently, the design operates with two clock domains, where the core operates at a four times higher frequency than the primary transform. The ASIC synthesis results for a TSMC 40nm standard-cells library indicate that our design can process UHD 4K videos at 120 frames per second while using an area of 69.68 Kgates, and with a power dissipation of 40.46 mW. When compared with related works, our design presented the lowest power dissipation and energy consumption per sample.

Design of Ternary Convolutional Code using Reconfigurable Architecture

The need for trustworthy and efficient digital data transmission and storage systems has increased in recent years. The demand to handle and store digital information has grown in the business sector, the military, and the government as a direct result of the widespread availability of high-speed, large-scale data networks. This requirement must be met in order for the design of these systems to keep up with the rapid pace set by communication and computer technologies. When trying to overcome the variable degradation in real time, one of the most important factors to consider is the dependability of the broadband communication channel. Therefore, the use of convolutional codes and other channel-coding strategies is an essential component of any broadband communication system. DSL, WLAN, and 3G standards all require different configurations of convolutional coding, each of which must achieve a specific level of coding performance despite operating at a unique data rate (constraint length and code rate) Therefore, from the perspective of channel-coding techniques, hardware implementations for the development of an encoder are essential. This encoder should be able to support multiple networks using a reconfigurability approach and should be able to function across a variety of standards. Additionally, flexibility and hardware performance should both be prioritised. This calls for forward error control coding with reconfigurable logic, which provides high-speed, low-power dynamically dedicated hardware architectures under a number of speed/power performance constraints at different time intervals that can function within a variety of channel conditions. The ternary computation system has a number of benefits, the most notable of which are the availability of a high data rate, improved spectral efficiency, enhanced coverage, and lower latency. As a result, there is a growing demand for ternary-based systems that use convolutional encoders as a result of developments in technology

Software-Hardware Co-Design for Energy-Efficient Continuous HealthMonitoring Via Task-Aware Compression.

Low power consumption associated with data transmission and processing of wearable/implantable devices is crucial to ensure the usability of continuous health monitoring systems. In this paper, we propose a novel health monitoring framework where the signal acquired is compressed in a task-aware manner to preserve task-relevant information at the sensor end with a low computation cost. The resulting compressed signals can be transmitted with significantly lower bandwidth, analyzed directly without a dedicated reconstruction process, or reconstructed with high fidelity. Also, we propose a dedicated hardware architecture with sparse Booth encoding multiplication and the 1-D convolution pipeline for the task-aware compression and the analysis modules, respectively. Extensive experiments show that the proposed framework is accurate, with a seizure prediction accuracy of 89.70 % under a signal compression ratio of 1/16. The hardware architecture is implemented on an Alveo U250 FPGA board, achieving a power of 0.207W at a clock frequency of 100MHz.

An Energy-Efficient Accelerator for Medical Image Reconstruction From Implicit Neural Representation

This work presents an energy-efficient accelerator for medical image reconstruction from implicit neural representation (INR). The accelerator implements an INR-based algorithm to deliver high-quality medical image reconstruction with arbitrary resolution from a compact implicit format. In particular, we propose a dedicated hardware architecture based on an optimized computation flow for the INR-based reconstruction algorithm, which co-designs data reuse and computation load. The proposed architecture takes in the coordinate of the intersection of three scans and outputs all the voxel intensities, minimizing the data movement between on-chip and off-chip. To validate the proposed accelerator, we build a proof-of-concept prototype demonstration system using field programmable gate array (FPGA). We also map our design to 40nm CMOS technology to measure the performance of the proposed accelerator. The implementation results show that, running at 400MHz, the proposed accelerator is capable of processing medical images with <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$256\times 256$ </tex-math></inline-formula> resolution in real-time at 26.3 frames per second (FPS), with a power consumption of only 795 mW. Comparison results show that the performance, as well as the energy efficiency of the proposed accelerator, outperforms the central processing unit (CPU)-based and graphic processing unit (GPU)-based implementations.

Recent Progress and Development of Hyperdimensional Computing (HDC) for Edge Intelligence

Brain-inspired Hyperdimensional Computing (HDC) is an emerging framework in low-energy designs for solving classification tasks at the edge. Unlike mainstream neural networks (NNs) with high computational complexity, HDC offers a fast training strategy and an energy-efficient inference mechanism by computing data in the high-dimensional space with high parallelism. However, the original computing method of HDC results in weak accuracy due to its simplicity. Therefore, many works have proposed algorithmic innovations to improve the accuracy of HDC, while maintaining its efficiency with dedicated hardware architectures. Given this period of rapid evolution, this review paper presents a comprehensive study of recent developments improving the accuracy-efficiency trade-off of HDC, covering both algorithm and hardware-level approaches. Besides, as the amount of data generated by the Internet of the Things (IoT) devices keeps increasing, edge-cloud collaboration has become a key area of research. The emerging federated learning (FL) has become a key enabler in overcoming security and privacy issues by exchanging locally trained models for global aggregation. However, the energy constraint of edge devices hinders them from the real-time training of NN models. Alternatively, adopting HDC as a local model can be a potential solution to alleviating this issue since the computational requirements of HDC are affordable to most edge devices. In addition, HDC-based coding for wireless communication has been gaining traction due to its noise robustness. Therefore, this paper also covers recent works on HDC-based edge-cloud collaborative learning. Finally, we conclude this paper by highlighting the possible applications and future directions of HDC.

A Flexible Architecture for Cryptographic Applications: ECC and PRESENT

This work presents a flexible/unified hardware architecture of Elliptic-curve Cryptography (ECC) and PRESENT for cryptographic applications. The features of the proposed work are (i) computation of only the point multiplication operation of ECC over for a 163-bit key generation, (ii) execution of only the variant of an 80-bit PRESENT block cipher for data encryption & decryption and (iii) execution of point multiplication operation (ECC algorithm) along with the data encryption and decryption (PRESENT algorithm). To establish an area overhead for the flexible design, dedicated hardware architectures of ECC and PRESENT are implemented in the first step, and a sum of their hardware area is computed. Then, the implementation of the proposed flexible architecture for ECC and PRESENT algorithms is presented. Implementation results regarding the area, clock cycles, latency, clock frequency, and power after the place-and-route level on Xilinx Virtex-5, Virtex-6, and Virtex-7 FPGA devices are presented. Hence, the implementation results and comparisons show that the proposed architecture suits applications demanding flexible implementation of cryptographic applications.

CHAMP: Channel Merging Process for Cost-Efficient Highly-Pruned CNN Acceleration

This paper presents an advanced offline scheduling scheme to improve the accelerator efficiency, especially for the highly-pruned convolutional neural networks (HP-CNNs). Based on the existing outlier-aware accelerator design, we demonstrate the efficiency drop of HP-CNN processing for the first time, and element-wise channel merging is proposed to make a dense processing sequence even for the highly-pruned model. The dedicated hardware architecture is also presented to process the merged channels with the minimum hardware-level overheads, improving the energy efficiency for handling HP-CNNs by preserving the hardware utilization. We further investigate the optimal size of accumulator and multiplexer, in addition to the number of merged channels, exploiting the attractive energy-performance trade-offs. As a result, unlike the practical HP-CNNs for the on-device solutions, the proposed method enhances the overall efficiency by up to 33% compared to the state-of-the-art schemes.

A Block PatchMatch-Based Energy-Resource Efficient Stereo Matching Processor on FPGA

This paper presents a field programmable gate array (FPGA)-based, high-performance and energy-resource efficient stereo matching processor. The proposed processor executes block-level PatchMatch-based stereo matching algorithm with a random search strategy to avoid estimation of all disparity levels. To take advantages of different block scales, a coarse-to-fine multi-scale propagation (MSP) scheme is proposed for label update. Based on that, a dedicated hardware architecture is further proposed to explore the benefit of the algorithm. Experimental results show that the proposed FPGA-based processor, running at 350MHz, achieves a peak performance of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1920\times 1080.165$ </tex-math></inline-formula> .7 frame per second (FPS) at 128 disparity levels with 3.35W power dissipation. The energy and resource efficiency of the proposed design outperforms state-of-the-art FPGA-based stereo matching processors. When disparity level increases to 256, the computing resource increment of the proposed design is much less than existing designs because random search instead of winner-takes-all (WTA) is utilized. Moreover, unlike existing dedicated stereo matching processors which output only disparity information, the proposed design is also capable of deriving plane slant. This information can be beneficial for follow-up tasks like 3D reconstruction.

Asynchronous Double-Frame-Exposure Binocular-Camera With Pixel-Level Pipeline Architecture for High-Speed Motion Tracking

The synchronous double-frame exposure (S-DFE), also called frame-straddling, has been widely applied for commercial video-based velocity measurements to provide high-speed velocimetry-flow images. Nevertheless, no low-cost, high-speed cameras offer the video quality that researchers, scientists, and industry demand. This brief presents an asynchronous DFE (A-DFE) method with a binocular camera for motion estimation, which captures double frames with the scalable time interval of the dual generic cameras. This time interval can significantly improve the upper limit of displacements of the motion, which is commonly solved by an expensive high-speed camera. A dedicated hardware architecture is designed to implement the perspective transformation and Wallis filter to eliminate the two cameras’ position mismatch and brightness variance. We build a high-speed Lucas-Kanade optical flow estimation system with A-DFE on the XILINX Spartan-7 FPGA platform with MT9V034 CMOS image sensors. The shortest time interval can reach 1.06 ns, which means a significant improvement of the upper limit of the measurement velocity. Except for the image sensor, the compact, fully parallel architecture can attain energy efficiency with 3.23 nJ/pixel on a low-cost FPGA device.

Efficient-Grad: Efficient Training Deep Convolutional Neural Networks on Edge Devices with Grad ient Optimizations

With the prospering of mobile devices, the distributed learning approach, enabling model training with decentralized data, has attracted great interest from researchers. However, the lack of training capability for edge devices significantly limits the energy efficiency of distributed learning in real life. This article describes Efficient-Grad, an algorithm-hardware co-design approach for training deep convolutional neural networks, which improves both throughput and energy saving during model training, with negligible validation accuracy loss. The key to Efficient-Grad is its exploitation of two observations. Firstly, the sparsity has potential for not only activation and weight, but gradients and the asymmetry residing in the gradients for the conventional back propagation (BP). Secondly, a dedicated hardware architecture for sparsity utilization and efficient data movement can be optimized to support the Efficient-Grad algorithm in a scalable manner. To the best of our knowledge, Efficient-Grad is the first approach that successfully adopts a feedback-alignment (FA)-based gradient optimization scheme for deep convolutional neural network training, which leads to its superiority in terms of energy efficiency. We present case studies to demonstrate that the Efficient-Grad design outperforms the prior arts by 3.72x in terms of energy efficiency.

Modern Video Coding: Methods, Challenges and Systems

With the increasing demand for digital video applications in our daily lives, video coding and decoding become critical tasks that must be supported by several types of devices and systems. This paper presents a discussion of the main challenges to design dedicated hardware architectures based on modern hybrid video coding formats, such as the High Efficiency Video Coding (HEVC), the AOMedia Video 1 (AV1) and the Versatile Video Coding (VVC). The paper discusses eachstep of the hybrid video coding process, highlighting the main challenges for each codec and discussing the main hardware solutions published in the literature. The discussions presented in the paper show that there are still many challenges to be overcome and open research opportunities, especially for the AV1 and VVC codecs. Most of these challenges are related to the high throughput required for processing high and ultrahigh resolution videos in real time and to energy constraints of multimedia-capable devices.

Floating-Point Inverse Square Root Algorithm Based on Taylor-Series Expansion

This brief describes a segmented structure to deal with inverse square root in floating-point digital calculation arithmetic, based on Taylor-Series expansion; it uses only the small number of their expansion terms to achieve a fast evaluation of these functions in high precision. Taylor-series expansions of the inverse square root are examined for several center points with their convergence ranges, and the inverse square root calculation algorithm trade-offs among accuracy, numbers of multiplications/additions/subtractions and LUT sizes are shown; the designer can choose the optimal algorithm for his digital inverse square root calculation, and build its conceptual dedicated hardware architecture design with the contents described here.

Class-specific early exit design methodology for convolutional neural networks

Convolutional Neural Network-based (CNN) inference is a demanding computational task where a long sequence of operations is applied to an input as dictated by the network topology. Optimisations by data quantisation, data reuse, network pruning, and dedicated hardware architectures have a strong impact on reducing both energy consumption and hardware resource requirements, and on improving inference latency. Implementing new applications from established models available from both academic and industrial worlds is common nowadays. Further optimisations by preserving model architecture have been proposed via early exiting approaches, where additional exit points are included in order to evaluate classifications of samples that produce feature maps with sufficient evidence to be classified before reaching the final model exit. This paper proposes a methodology for designing early-exit networks from a given baseline model aiming to improve the average latency for a targeted subset class constrained by the original accuracy for all classes. Results demonstrate average time saving in the order of 2.09× to 8.79× for dataset CIFAR10 and 15.00× to 20.71× for CIFAR100 for baseline models ResNet-21, ResNet-110, Inceptionv3-159, and DenseNet-121.

FPGA Implementation of 3-bit Quantized CNN for Semantic Segmentation

Semantic segmentation is a task of inputting an image and performing category classification for each pixel. Semantic segmentation by CNN has high accuracy but its calculation using floating-point numbers consumes a large amount of power. We adopted UNET as the semantic segmentation CNN and improved it for FPGA implementation. We quantized both weights and activations of the network up to 3-bit. Then, we devised a dedicated hardware architecture for the quantized CNN and implemented it on an FPGA. This circuit uses only internal memory to perform forward propagation calculations, that eliminates high-power external memory accesses. This circuit is a stall-free pixel-by-pixel pipeline, and performs 8 rows, 16 input channels, 16 output channels, 3 by 3 pixels convolution calculations in parallel. The convolution calculation performance at an operating frequency of 300 MHz is 11 TOPs/s.

FPGA Implementation of Particle Filters for Robotic Source Localization

Particle filtering is very reliable in modelling non-Gaussian and non-linear elements of physical systems, which makes it ideal for tracking and localization applications. However, a major drawback of particle filters is their computational complexity, which inhibits their use in real-time applications with conventional CPU or DSP based implementation schemes. The re-sampling step in the particle filters creates a computational bottleneck since it is inherently sequential and cannot be parallelized. This paper proposes a modification to the existing particle filter algorithm, which enables parallel re-sampling and reduces the effect of the re-sampling bottleneck. We then present a high-speed and dedicated hardware architecture incorporating pipe-lining and parallelization design strategies to supplement the modified algorithm and lower the execution time considerably. From an application standpoint, we propose a novel source localization model to estimate the position of a source in a noisy environment using the particle filter algorithm implemented on hardware. The design has been prototyped using Artix-7 field-programmable gate array (FPGA), and resource utilization for the proposed system is presented. Further, we show the execution time and estimation accuracy of the high-speed architecture and observe a significant reduction in computational time. Our implementation of particle filters on FPGA is scalable and modular, with a low execution time of about 5.62 μs for processing 1024 particles (compared to 64 ms on Intel Core i7-7700 CPU with eight cores clocking at 3.60 GHz) and can be deployed for real-time applications.

Dedicated hardware architecture for localizing iris in VW images

This study presents dedicated hardware for iris localization that can be used as a coprocessor in the development of real-time and low-cost embedded iris biometric systems. Though the hardware architecture is described for iris localization in the visible wavelength (VW) images, the concept used can be applied to near infrared (NIR) images as well. In general, the architecture can be used for a class of iris localization algorithms based on the edge-map generation and circular Hough transform (CHT). The architecture presented here generates the edge-maps for limbic and pupil boundary detection using median filtering followed by Sobel edge detection; however, an additional reflection removal module is used for pupil boundary detection. Further, the CHT hardware module detects circle in each edge-map. The proposed architecture was implemented in programmable logic of the Zynq-7000 SoC device from Xilinx. This hardware implementation gives an iris localization accuracy of 98.43% and average processing time of 5.148 ms for UBIRIS.v1 VW database images (200 × 150 pixel). The algorithm used is suitable for less unconstrained and frontal-view iris images captured with subjects’ active participation; however, the images may contain non-ideal issues such as reflection and occlusion by eyelids and eyelashes.

A 975-mW Fully Integrated Genetic Variant Discovery System-on-Chip in 28 nm for Next-Generation Sequencing

This article presents the first dedicated system-onchip (SoC) that supports full data analysis workflow for genetic variant discovery for next-generation sequencing (NGS). The SoC implements four major steps: preprocessing, short-read mapping, haplotype calling, and variant calling. By adopting the sBWT-based short read mapping and Genome Analysis ToolKit haplotype caller-based variant calling algorithms, this work achieves a comparable precision as the software solutions. The dedicated hardware architecture achieves high parallelism with low hardware complexity. In this SoC, the multitask sorting engine and the dynamic programming processing engine are proposed for essential computing tasks for NGS data analysis. An ARC processor is integrated to facilitate IO interfaces, including a DDR3 PHY. Fabricated in a 28-nm technology, the chip area is 12 mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> . It dissipates 975 mW at a clock frequency of 400 MHz from a 0.9-V supply. The SoC is able to complete variant discovery for the whole human genome within 37 min. Compared with an optimized graphic processing unit solution, this work is 66× faster and also achieves an 1.59 × 10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> times higher energy efficiency and 3086× higher area efficiency. Verified with the FDA standardized benchmark, the SoC achieves a precision of 99.6%. The prototype system demonstrates the analysis procedure in real time for on-site DNA sequencing.

Power-Efficient Approximate Newton–Raphson Integer Divider Applied to NLMS Adaptive Filter for High-Quality Interference Cancelling

The division datapath is undoubtedly the most complex operation in a wide range of digital signal processing applications, such as in adaptive filtering algorithms. This paper proposes an optimized and approximate integer divider hardware architecture, based on the Newton–Raphson algorithm combining both fixed-point dynamic range and truncation techniques, to speed up that operation. Adaptive filters have been much studied over time, as they comprise one of the most challenging fields in signal processing. This work presents dedicated hardware architectures based on normalized least mean square adaptive filtering algorithms for the power line harmonics interference cancelling. The hardware architectures are based on 2’s complement representation and were described in VHDL and synthesized into a 65 nm CMOS dedicated ASIC. Our results show that the increased approximation level of Newton–Raphson divider approximation presents up to 223 times less power dissipation than the baseline version without our optimization and approximations, providing up to 93 times of power dissipation savings in the complete interference canceller system.