On-line Arithmetic Research Articles

ECHO: Energy-Efficient Computation Harnessing Online Arithmetic—An MSDF-Based Accelerator for DNN Inference

Deep neural network (DNN) inference demands substantial computing power, resulting in significant energy consumption. A large number of negative output activations in convolution layers are rendered zero due to the invocation of the ReLU activation function. This results in a substantial number of unnecessary computations that consume significant amounts of energy. This paper presents ECHO, an accelerator for DNN inference designed for computation pruning, utilizing an unconventional arithmetic paradigm known as online/most significant digit first (MSDF) arithmetic, which performs computations in a digit-serial manner. The MSDF digit-serial computation of online arithmetic enables overlapped computation of successive operations, leading to substantial performance improvements. The online arithmetic, coupled with a negative output detection scheme, facilitates early and precise recognition of negative outputs. This, in turn, allows for the timely termination of unnecessary computations, resulting in a reduction in energy consumption. The implemented design has been realized on the Xilinx Virtex-7 VU3P FPGA and subjected to a comprehensive evaluation through a rigorous comparative analysis involving widely used performance metrics. The experimental results demonstrate promising power and performance improvements compared to contemporary methods. In particular, the proposed design achieved average improvements in power consumption of up to 81%, 82.9%, and 40.6% for VGG-16, ResNet-18, and ResNet-50 workloads compared to the conventional bit-serial design, respectively. Furthermore, significant average speedups of 2.39×, 2.6×, and 2.42× were observed when comparing the proposed design to conventional bit-serial designs for the VGG-16, ResNet-18, and ResNet-50 models, respectively.

Two applications of the spectrum of numbers

Let the base $${\beta}$$ be a complex number, $${|\beta| > 1}$$ , and let $${A \subset \mathbb{C}}$$ be a finite alphabet of digits. The A-spectrum of $${\beta}$$ is the set $${{S}_{A}(\beta) = {\{\Sigma^{n}_{k=0} {a}_{k}\beta^{k} | n\in \mathbb{N}, {a}_{k} \in A\}}}$$ . We show that the spectrum $${{S}_{A}(\beta)}$$ has an accumulation point if and only if 0 has a particular $${(\beta, A)}$$ -representation, said to be rigid. The first application is restricted to the case that $${\beta > 1}$$ and the alphabet is A = {−M, . . . , M}, $${{M \geq}}$$ 1 integer. We show that the set $${{Z}_{\beta, M}}$$ of infinite $${(\beta, A)}$$ -representations of 0 is recognizable by a finite Buchi automaton if and only if the spectrum $${{S}_{A}(\beta)}$$ has no accumulation point. Using a result of Akiyama–Komornik and Feng, this implies that $${{Z}_{\beta, M}}$$ is recognizable by a finite Buchi automaton for any positive integer $${M \geq\lceil {\beta\rceil-1}}$$ if and only if $${{\beta}}$$ is a Pisot number. This improves the previous bound $${M \geq \lceil \beta\rceil}$$ . For the second application the base and the digits are complex. We consider the on-line algorithm for division of Trivedi and Ercegovac generalized to a complex numeration system. In on-line arithmetic the operands and results are processed in a digit serial manner, starting with the most significant digit. The divisor must be far from 0, which means that no prefix of the $${(\beta,A)}$$ -representation of the divisor can be small. The numeration system $${(\beta,A)}$$ is said to allow preprocessing if there exists a finite list of transformations on the divisor which achieve this task. We show that $${(\beta,A)}$$ allows preprocessing if and only if the spectrum $${{S}_{A}(\beta)}$$ has no accumulation point.

Algorithms for Multiplierless Multiple Constant Multiplication in Online Arithmetic

Online arithmetic operators offer advantages of reduction in resource utilization and interconnection complexity besides providing pipelining at digit level. Multiplierless constant coefficient multiplication using the shift-and-add technique is widely used in digital signal processing applications. This paper proposes a novel bit serial adaptation of the parallel shift-and-add algorithm to online arithmetic. The proposed multipliers use right shifts instead of the traditional left shifts resulting in causal online implementations. Graph-based and hybrid algorithms are developed for the estimation of the distance of a constant from a set of constants in terms of the number of additions and for the synthesis of online multiple constant multipliers under area and online delay constraints. The computational complexity of the algorithms is determined. Results of implementation on randomly generated constant sets and FIR filter instances show substantial improvements in the number of operations required using the distance heuristic. Further, it is shown that the proposed techniques and algorithms result in significant savings in resource utilization, logic depth, and clock frequency compared to parallel and digit-serial algorithms.

Deeply Pipelined Digit-Serial LDPC Decoding

Highly parallel VLSI implementations of low-density parity-check (LDPC) decoders have a large number of interconnections, which can result in designs with low logic density. Bit-serial architectures have been developed that reduce the number of wires needed, however, they do not fully realize the potential for deeply pipelined serial data processing. Digit- online arithmetic allows operations to be performed in a serial, digit-by-digit manner, making it ideal for use in implementing a digit-serial LDPC decoder. Digit-online circuits for the primitive operations required for an offset min-sum LDPC decoder are simple, and allow deep pipelining at the digit level. A new hardware architecture for LDPC decoding is demonstrated, using redundant number systems for the internal representation of values. We present post-layout decoder results for the (2048, 1723) 10GBASE-T LDPC code in a general-purpose 65 nm CMOS technology. The decoder requires a core area of 10.89 mm and operates at a clock frequency of 980 MHz. The decoder can simultaneously decode two 4-bit frames at 41.8 Gbit/s or one 10-bit frame at 20.9 Gbit/s.

Reconfigurable VBSME Architecture Using RBSAD

This paper presents an architecture which is capable of processing variable block size motion estimation (VBSME) and which is able to apply pixel precision reduction techniques in a reconfigurable way. The design has been carried out by using online arithmetic, which allows to process all motion vectors of a block in just one iteration. The system has been implemented on FPGA and just requires 7724 slices, reaching a performance of 55 4CIF frames per second (fps) in full precision and of 72 with 4 bit precision. Results for different search areas 31× 31, 32× 32, and 46× 46 are presented. Using 4bit precision real time processing for HDTVp is achieved. Thanks to the reduced cost and high performance, this architecture is perfect for mobile devices.

A Low Cost Architecture for Variable Block Size Motion Estimation

Variable block size motion estimation is adopted in MPEG-4 AVC/H.264. This paper presents a new VLSI and FPGA architecture using full search block matching algorithm and online arithmetic. Several ways for data refreshing are described. There is not any increment in the number of clock cycles to process all sub-block formats. Only 54K gates are used, allowing to implement this architecture in devices with low hardware requirements. Moreover, low power consumption is obtained. A qualitative analysis of other designs is reported. Early termination of SAD calculation is analysed. Real-time video processing can be achieved for HDTV using early termination or increasing the parallelism.

Truncated Online Arithmetic with Applications to Communication Systems

Truncation in digit-precision is a very important and common operation in embedded system design for bounding the required finite precision and for area-time-power savings. In this paper, we present the use of online arithmetic to provide truncated computations with communication systems as one of the applications. In contrast to truncation in conventional arithmetic, online arithmetic can truncate dynamically and produce both area and time benefits due to the digit-serial nature of computations. This is of great advantage in communication systems where the precision requirements can change dynamically with the environment. While truncation in conventional arithmetic can have significant truncation errors, especially when the output precision is less than the input precision, the redundancy and most significant digit first nature of online arithmetic restricts the truncation error to only the least significant digit of the truncated result. As an application that uses significant truncation in precision, a code matched filter detector for wireless systems is designed using truncated online arithmetic. The detector can provide both hard decisions and soft(er) decisions dynamically as well as interface with other conventional arithmetic circuits or act as a DSP coprocessor. Thus, optimized communication receivers with coexisting conventional arithmetic for saturation and online arithmetic for truncation can now be built. The truncated online arithmetic detector was also verified with a VLSI implementation in an AMI 0.5 mu MOSIS tiny chip process

SAD computation based on online arithmetic for motion estimation

Block-based motion estimation is one of the critical tasks in today's video compression standards such as H.26x, MPEG-1, -2 and -4. Most of the block-based motion estimation algorithms are based on computing the sum of absolute differences (SAD) between corresponding elements in the candidate and reference blocks. In this paper, an field-programmable gate-array (FPGA) design is proposed for rapidly computing the minimum SAD. Two goals are achieved due to the use of online arithmetic (OLA): it is possible to implement a full 16×16 macroblock SAD in a single FPGA device; and it allows us to speed up computation by early termination of the SAD calculation when the candidate involved is bigger than the current reference SAD. Reconfigurable devices enable us to change 8×8 or 16×16 pixels per block quickly and easily. For a 16×16 SAD unit 1945 look-up tables (LUTs) are required at 425 MHz. A comparison with other related works is provided.

A design methodology for networks of online modules and its application to the Levinson-Durbin algorithm

The use of online arithmetic was often proposed for hardware implementations of complex digital-signal processing (DSP) algorithms. However, several important issues in the design process of such algorithms using online arithmetic are rarely discussed in the literature. This paper presents these issues and provides a methodology to analyze the behavior of networks of online arithmetic modules performing serial computation over fixed-point numbers. The methodology is presented, applied in several examples, and finally used to design an efficient field programmable gate arrays implementation of the Levinson-Durbin algorithm in an application of the Yule-Walker power spectrum estimation. The methodology can be applied to other algorithms as well and it simplifies the task of designing and verifying a network of online modules. The experimental results show the advantages of online arithmetic in the design of complex DSP algorithms.

Motion estimation using MSD-first processing

A new most-significant-digit-first (MSD-first) motion estimation (ME) algorithm based on online arithmetic is proposed in the paper, together with its associated architecture. High-performance ME architectures are crucial for high-quality video applications with high resolution and frame rate, such as HDTV, DVD-video recorders and digital camcorders. The proposed algorithm decomposes the summation of absolute difference (SAD) and the comparison operations of full search block matching (FSBM) into digit level. The digit-level comparisons are interleaved into the separated SAD operations to distinguish the motion vector (MV) as early as possible. It operates in the MSD-first manner and eliminates redundant operations for less significant digits, while still extracting the exact MV of the selected search algorithm. The authors also demonstrate the online arithmetic designs for SAD and comparison, which are two primitive operations in the proposed algorithm. The proposed method saves 47.4% to 64.3% SAD computations of FSBM. In the implementation, a 4×4 array processor of the proposed ME using online arithmetic has a 2.84 ns critical path and 1510 gates with 0.35 μm 1P4 M CMOS cell library. It supports 83 mega 4×4 block matching per second and also reduces the ME operations of other fast search algorithms.

On-line arithmetic for real-time control of microsystems

The integration of microcontrollers within mechanical systems is a current trend. However, decreasing the size of the system and satisfying higher precision requirements make it necessary to reevaluate the common signal processing techniques for controller implementations, because limited controller size, computation speed, and power consumption become major topics. In this paper, we demonstrate that serial computations with the most significant digits first, that is, online arithmetic, offer an important potential for real-time control. They enable a combination of traditional functions, such as analog-to-digital converters and control data computations. This leads to very efficient controller implementations with small size, high speed, and low power consumption. After a brief description of the requirements and challenges of microsystem controller design, the use of online arithmetic for real-time control is proposed. A short introduction to online arithmetic is given and control-specific implementation guidelines are presented and finally applied to a simple test system.

Software for high radix on-line arithmetic

High radix on-line arithmetic provides an efficient method for performing variable-precision arithmetic. It can be implemented on conventional microprocessors using sequences of three operand instructions. This paper presents software support for high radix on-line arithmetic. This software includes emulation modules for high radix operations, and a precision analysis program for setting the input and intermediate variable tolerances necessary for guaranteed result accuracy over a specified domain.

Algorithm-architecture co-design by example: a coprocessor for on-line arithmetic

Domain specific architectures gain more and more attention in high performance applications, when general purpose processors are not capable of achieving the desired throughput. The algorithms to be implemented strongly influence the architecture of the design and vice versa. A systematic approach based on provably correct transformations, herein called Algorithm-Architecture Co-design, will be demonstrated on the design of a processor for long integer arithmetic.

A High Radix On-line Arithmetic for Credible and Accurate Computing

A High Radix On-line Arithmetic for Credible and Accurate Computing

A systematic methodology for the design of high performance recursive digital filters

A systematic design methodology is described for the rapid derivation of VLSI architectures for implementing high performance recursive digital filters, particularly ones based on most significant digit (msd) first arithmetic. The method has been derived by undertaking theoretical investigations of msd first multiply-accumulate algorithms and by deriving important relationships governing the dependencies between circuit latency, levels of pipelining and the range and number representations of filter operands. The techniques described are general and can be applied to both bit parallel and bit serial circuits, including those based on on-line arithmetic. The method is illustrated by applying it to the design of a number of highly pipelined bit parallel IIR and wave digital filter circuits. It is shown that established architectures, which were previously designed using heuristic techniques, can be derived directly from the equations described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Improving average delay for on-linearithmetic algorithms

Using normalised input directly for on-line arithmetic does not always lead to minimal on-line delay. The author shows how shifting some inputs can reduce the average delay. >

Conventional and on-line arithmetic designs for high-speed recursive digital filters

Designs using conventional and on-line arithmetic for performing the recursive computations of Second-Order Direct-Form IIR filters are implemented in a 0.7μ HCMOS gate array technology. The new conventional (bit-parallel) design using two levels of scattered lookahead achieves a rate of 104 Msamples/second. The design eliminates carry propagate addition from the recursive loop and uses radix-4 recoding to reduce the number of partial products. The on-line design is an improved radix-4 multiply-add (MA) module. Two arrays based on the on-line MA module are developed to achieve rates of 128 Msamples/second with two levels of scattered lookahead. Performance and cost comparisons of the designs provide insights into the most appropriate designs to be used for a given word length.

Some characterizations of functions computable in on-line arithmetic

After a short introduction to on-line computing, we prove that the functions computable in on-line by a finite automaton are piecewise affine functions whose coefficients are rational numbers (i.e., the functions f(x)=ax+b, or f(x,y)=ax+by+c where a, b, and c are rational). A consequence of this study is that multiplication, division and elementary functions of operands of arbitrarily long length cannot be performed using bounded-size operators.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A new model for on-line arithmetic with an application to the reciprocal calculation

A new model for on-line arithmetic functions is presented. It differs from the current models that in each iteration step the exact function value is used to generate the on-line digits. From this model the on-line properties of a large class of arithmetic functions can be determined. In general, the on-line properties of an arithmetic function derived from the model cannot be improved by any iterative approximation algorithm for that arithmetic function. The on-line properties of a number of standard arithmetic functions are given. It is shown that the above model of on-line computation can be easily implemented by means of a table look-up system. Furthermore, a table implementation can be used to start the on-line computation of an iterative approximation method. This is shown by an example, the reciprocal calculation, where the combination of a seed table and an adapted Newton-Raphson iteration method leads to a system with a low on-line delay and fast cycle times. The algorithm works for normalized, quasi-normalized, and pseudo-normalized numbers and can therefore be applied to chained on-line computations.

Redundant logarithmic arithmetic

A number system that offers advantages in some situations over conventional floating point and sign/logarithmic number systems is described. Redundant logarithmic arithmetic, like conventional logarithmic arithmetic, relies on table lookups to make the arithmetic unit simpler than an equivalent floating point unit. The cost of 32 bit subtraction in a redundant logarithmic number system is lower than previously published logarithmic subtraction methods. The total memory requirement for a 29-bit redundant logarithmic unit is 16 K words compared to 22 K words by the best previously published conventional sign logarithm unit, assuming similar addition techniques are employed. A redundant logarithmic number system can be implemented with online arithmetic, which would be impractical for a conventional sign logarithm number system. The disadvantages of redundant arithmetic are typical of redundant number systems. First, the redundancy doubles the storage requirements for data values. Second, the representation can become ill-conditioned, especially as a result of iterated multiplications. Third, division and square root operations are more difficult to implement in redundant logarithmic arithmetic.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>