Number Of Nonzero Digits Research Articles

FloatSD: A New Weight Representation and Associated Update Method for Efficient Convolutional Neural Network Training

In this paper, we propose a floating-point signed digit (FloatSD) format for convolutional neural network (CNN) weight representation and its update method during CNN training. The number of non-zero digits in a weight can be as few as two during the forward and backward passes of the CNN training, reducing the convolution multiplication to addition of two shifted multiplicands (partial products). Furthermore, the mantissa field and the exponent field of neuron activations and gradients during training are also quantized, leading to floating-point numbers represented by eight bits. We tested the FloatSD method using three popular CNN applications, namely, MNIST, CIFAR-10, and ImageNet. These three CNNs were trained from scratch using the conventional 32-bit floating-point arithmetic and the FloatSD weight representation, 8-bit floating-point numbers for activations and gradients, and half-precision 16-bit floating-point accumulation. We obtained FloatSD accuracy results very close to or even better than those trained in 32-bit floating-point arithmetic. Finally, the proposed method not only significantly reduces the computational complexity for CNN training but also achieves memory capacity and bandwidth saving of about three quarters, demonstrating its effectiveness in the low-complexity implementation of CNN training.

An Effective Allocation of Non-Zero Digits for CSD Coefficient FIR Filters Using 0-1PSO

In this paper, a novel method for effective allocation of non-zero digits in design of CSD (Canonic Signed Digit) coefficient FIR (Finite Impulse Response) filters is proposed. The design problem can be formulated as a mixed integer linear programming problem, which is well-known as a NP-hard problem. Recently, a heuristic approach using the PSO (Particle Swarm Optimization) for solving the problem has been proposed, in which the maximum number of non-zero digits is limited in each coefficient. On the other hand, the maximum number of non-zero digits is limited in total in the proposed method and the 0-1PSO is applied. It enables an effective allocation of nonzero digits and provides a good design. Several examples are shown to present an efficiency of the proposed method.

Optimality of the Width-w Non-adjacent Form: General Characterisation and the Case of Imaginary Quadratic Bases

Efficient scalar multiplication in Abelian groups (which is an important operation in public key cryptography) can be performed using digital expansions. Apart from rational integer bases (double-and-add algorithm), imaginary quadratic integer bases are of interest for elliptic curve cryptography, because the Frobenius endomorphism fulfils a quadratic equation. One strategy for improving the efficiency is to increase the digit set (at the prize of additional precomputations). A common choice is the width\nbd-$w$ non-adjacent form (\wNAF): each block of $w$ consecutive digits contains at most one non-zero digit. Heuristically, this ensures a low weight, i.e.\ number of non-zero digits, which translates in few costly curve operations. This paper investigates the following question: Is the \wNAF{}-expansion optimal, where optimality means minimising the weight over all possible expansions with the same digit set? The main characterisation of optimality of \wNAF{}s can be formulated in the following more general setting: We consider an Abelian group together with an endomorphism (e.g., multiplication by a base element in a ring) and a finite digit set. We show that each group element has an optimal \wNAF{}-expansion if and only if this is the case for each sum of two expansions of weight 1. This leads both to an algorithmic criterion and to generic answers for various cases. Imaginary quadratic integers of trace at least 3 (in absolute value) have optimal \wNAF{}s for $w\ge 4$. The same holds for the special case of base $(\pm 3\pm\sqrt{-3})/2$ and $w\ge 2$, which corresponds to Koblitz curves in characteristic three. In the case of $\tau=\pm1\pm i$, optimality depends on the parity of $w$. Computational results for small trace are given.

On the g-Ary Expansions of Apéry, Motzkin, Schröder and Other Combinatorial Numbers

Let g ≥ 2 be an integer and let (un)n≥1 be a sequence of integers which satisfies a relation un+1 = h(n)un for a rational function h(X). For example, various combinatorial numbers as well as their products satisfy relations of this type. Here, we show that under some mild technical assumptions the number of nonzero digits of un in base g is large on a set of n of asymptotic density 1. We also extend this result to a class of sequences satisfying relations of second order un+2 = h1(n)un+1 + h2(n)un with two nonconstant rational functions \({h_1(X), h_2(X) \in {\mathbb Q} [X]}\). This class includes the Apery, Delannoy, Motzkin, and Schroder numbers.

Optimal Design of CSD Coefficient FIR Filters Subject to Number of Nonzero Digits

In a hardware implementation of FIR(Finite Impulse Response) digital filters, it is desired to reduce a total number of nonzero digits used for a representation of filter coefficients. In general, a design problem of FIR filters with CSD(Canonic Signed Digit) representation, which is efficient one for the reduction of numbers of multiplier units, is often considered as one of the 0-1 combinational problems. In such the problem, some difficult constraints make us prevent to linearize the problem. Although many kinds of heuristic approaches have been applied to solve the problem, the solution obtained by such a manner could not guarantee its optimality. In this paper, we attempt to formulate the design problem as the 0-1 mixed integer linear programming problem and solve it by using the branch and bound technique, which is a powerful method for solving integer programming problem. Several design examples are shown to present an efficient performance of the proposed method.

Exponential sums and prime divisors of sparse integers

We obtain a new lower bound on the number of prime divisors of integers whose g-ary expansion contains a fixed number of nonzero digits.

Design of Multiplierless Lattice QMF: Structure and Algorithm Development

In this brief, efficient multiplierless design of lattice quadrature mirror filter bank is presented. Previous work by the authors has shown that splitting each lattice stage into cascade of subrotations results in larger stopband attenuation of filter than the conventional direct quantization. This brief extends the work further by exploiting the subrotations which yield more flexible sum of signed powers-of-two quantization. This enables us to find more possible discrete representations, and hence to reduce the quantization error. Also, an algorithm for the efficient gathering of candidate discrete coefficients is developed, based on the trellis-based searching approach. It substantially alleviates the overheads of optimization program, especially when the wordlengths and the number of nonzero digits are large. Several design examples are provided to show that the proposed structure with the candidate gathering algorithm provides improved frequency response.

Convergents and irrationality measures of logarithms

We prove new irrationality measures with restricted denominators of the form \mathrm{d}_{\lfloor\nu m\rfloor}^s B^m (where B, m \in\mathbb{N}, \nu > 0 , s\in\{0,1\} and \mathrm{d}_m=\textup{lcm}\{1,2, \ldots, m\} ) for values of the logarithm at certain rational numbers r>0 . In particular, we show that such an irrationality measure of \log(r) is arbitrarily close to 1 provided r is sufficiently close to 1. This implies certain results on the number of non-zero digits in the b -ary expansion of \log(r) and on the structure of the denominators of convergents of \log(r) . No simple method for calculating the latter is known. For example, we show that, given integers a,c\ge 1 , for all large enough b, n , the denominator q_n of the n -th convergent of \log(1\pm a/b) cannot be written under the form \mathrm d_{\lfloor\nu m\rfloor}^s (bc)^m : this is true for a=c=1 , b \ge 12 when s=0 , resp. b \ge 2 when s=1 and \nu=1 . Our method rests on a detailed diophantine analysis of the upper Padé table ([p/q])_{p\ge q\ge 0} of the function \log(1-x) . Finally, we remark that worse results (of this form) are currently provable for the exponential function, despite the fact that the complete Padé table ([p/q])_{p, q\ge 0} of \exp(x) and the convergents of \exp(1/b) , for \vert b\vert \ge 1 , are well-known, for example.

Hamming weight pyramid – A new insight into canonical signed digit representation and its applications

Signed-power-of-two terms are widely used in design automation algorithms for digital filter synthesis and optimization, linear transformation and other multiple constant multiplication problems. In these applications, the computation efficiency or solution quality tends to degrade with the number of nonzero digits in the signed digit representation of the a priori fixed coefficients. This paper provides a new perspective to interpret the hamming weights of fixed-point coefficients represented in signed-power-of-two terms with minimal number of nonzero digits, called the minimal signed digit (MSD) representation. A new hamming weight pyramid (HWP) is proposed to succinctly compress the information about the distribution of the hamming weights of canonical signed digit (CSD) representation in a visually appealing manner for analysis and synthesis. CSD is a unique and popularly used subset of the general MSD representation. Many interesting properties of CSD are uncovered in this regularly structured HWP. These properties are exploited to develop a novel and elegant algorithm for the direct conversion of decimal number to CSD representation. We also show that the HWP can also be employed to overcome the limit imposed on the word length of the coefficients for the reduced adder graph (RAG) algorithm and filter coefficient synthesis.

A Simple Gray Code to List All Minimal Signed Binary Representations

A signed binary representation (SBR) of an integer $N$ is a string $a_b\cdots a_2a_1a_0$ over the alphabet $\{-1,0,1\}$ such that $N = \sum_{i=0}^b a_i2^i$. An SBR of an integer $N$ is said to be minimal if the number of nonzero digits is minimum. In this paper, we describe a simple 3-close Gray code for listing all minimal SBRs of an integer $N$. The algorithm is implemented to run in constant amortized time. In addition, we identify the values for $N$ that have the maximum number of minimal SBRs given the length of the binary representation of $N$.

Fast modular multi-exponentiation using modified complex arithmetic

Modular multi-exponentiation ∏ i = 1 n M i E i ( mod N ) is a very important but time-consuming operation in many modern cryptosystems. In this paper, a fast modular multi-exponentiation is proposed utilizing the binary-like complex arithmetic method, complement representation method and canonical-signed-digit recoding technique. By performing complements and canonical-signed-digit recoding technique, the Hamming weight (number of 1’s in the binary representation or number of non-zero digits in the binary signed-digit representations) of the exponents can be reduced. Based on these techniques, an algorithm with efficient modular multi-exponentiation is proposed. For modular multi-exponentiation, in average case, the proposed algorithm can reduce the number of modular multiplications (MMs) from 1.503 k to 1.306 k, where k is the bit-length of the exponent. We can therefore efficiently speed up the overall performance of the modular multi-exponentiation for cryptographic applications.

On the Number of Optimal Base 2 Representations of Integers

We study representations of integers n in binary expansions using the digits 0,±1. We analyze the average number of such representations of minimal weight (= number of non-zero digits). The asymptotic main term of this average involves a periodically oscillating function, which is analyzed in some detail. The main tool is the construction of a measure on [?1,1], which encodes the number of representations.

Minimal weight digit set conversions

We consider the problem of recoding a number to minimize the number of nonzero digits in its representation, that is, to minimize the weight of the representation. A general sliding window scheme is described that extends minimal binary sliding window conversion to arbitrary radix and to encompass signed digit sets. This new conversion expresses a number of known recoding techniques as special cases. Proof that this scheme achieves minimal weight for a given digit set is provided and results concerning the theoretical average and worst-case weight are derived.

Prime divisors of sparse integers

We obtain a lower bound on the number of prime divisors of integers whose g-ary expansion contains a fixed number of nonzero digits.

The Number of Non-Zero Digits of n!

AbstractLet b be an integer with b > 1. In this note, we prove that the number of non-zero digits in the base b representation of n! grows at least as fast as a constant, depending on b, times log n.

Nuclei of normal rational curves

Ak-nucleus of a normal rational curve inPG(n, F) is the intersection over allk-dimensional osculating subspaces of the curve (k e {−1,0,...,n− 1}). It is well known that for characteristic zero all nuclei are empty. In case of characteristicp}>0 and #F≥n the number of non-zero digits in the representation ofn+ 1 in basep equals the number of distinct nuclei. An explicit formula for the dimensions ofk-nuclei is given for #F=F≥k+ 1.

On the asymptotic existence of complex Hadamard matrices

Let N = N(q) be the number of nonzero digits in the binary expansion of the odd integer q. A construction method is presented which produces, among other results, a block circulant complex Hadamard matrix of order 2αq, where α ≥ 2N - 1. This improves a recent result of Craigen regarding the asymptotic existence of Hadamard matrices. We also present a method that gives complex orthogonal designs of order 2α+1q from complex orthogonal designs of order 2α. We also demonstrate the existence of a block circulant complex Hadamard matrix of order 2βq, where © 1997 John Wiley & Sons, Inc. J Combin Designs 5:319–327, 1997

On the asymptotic existence of complex Hadamard matrices

Let N = N(q) be the number of nonzero digits in the binary expansion of the odd integer q. A construction method is presented which produces, among other results, a block circulant complex Hadamard matrix of order 2αq, where α ≥ 2N - 1. This improves a recent result of Craigen regarding the asymptotic existence of Hadamard matrices. We also present a method that gives complex orthogonal designs of order 2α+1q from complex orthogonal designs of order 2α. We also demonstrate the existence of a block circulant complex Hadamard matrix of order 2βq, where © 1997 John Wiley & Sons, Inc. J Combin Designs 5:319–327, 1997

Signed groups, sequences, and the asymptotic existence of Hadamard matrices

We use the newly developed theory of signed groups and some known sequences with zero autocorrelation to derive new results on the asymptotic existence of Hadamard matrices. New values of t are obtained such that, for any odd number p, there exists an Hadamard matrix of order 2 t p. These include: t = 2 N, where N is the number of nonzero digits in the binary expansion of p, and t = 4⌈ 1 6 log 2( (p − 1) 2 ⌉ + 2 . Both numbers improve on all previous general results, but neither uses the full power of our method. We also discuss some of the implications of our method in terms of signed group Hadamard matrices and signed group weighing matrices: There exists a circulant signed group Hadamard matrix of every even order n, using a suitable signed group. This result stands in striking contrast to the known results for Hadamard matrices and complex Hadamard matrices, and the circulant Hadamard matrix conjecture. Signed group weighing matrices of even order n always exist, with any specified weight w ⩽ n.

On modular weight and cyclic nonadjacent forms for arithmetic codes (Corresp.)

In this correspondence we give a new definition of arithmetic weight (relative to any modulus m > 0 and any radix r \geq 2 ), which has two very useful properties, i) It is defined for all integers and is invariant under congruence modulo m . ii) It yields a metric for all moduli m . In the important cases m = r^n, m = r^n - 1 , and m = r^n + 1 , it coincides with the weight of Rao and Garcia, and thus acts as a proper measure of errors for these moduli. Due to these properties, we are able to obtain conceptually simpler proofs of several known (as well as some new) results on the computation of distance in cyclic AN-codes. Along these same lines we introduce the notion of modular-cyclic nonadjacent form (NAF) when m = r^n - 1 . For moduli of this type, we show that every integer has a modular-cyclic NAF; the number of nonzero digits is the weight. It is a curious fact that a modular version of the well-known Chang/Tsao-Wu algorithm (for computing the NAF) turns out to yield the modular-cyclic NAF directly.