Floating Point Number Representation Research Articles

The Laplace Microarchitecture for Tracking Data Uncertainty

This article presents Laplace, a microarchitecture for tracking machine representations of probability distributions paired with architectural state. Laplace uses in-processor distribution representations, which are approximations of probability distributions just as floating-point number representations are approximations of real-valued numbers. The article presents two sets of instruction set architecture (ISA) extensions to 1) provide a mechanism to initialize distributional information in the microarchitecture; and 2) to allow applications to query statistics of the distributional information without exposing the uncertainty representations above the ISA. Unlike existing methods for uncertainty tracking, which require software to be rewritten in a domain-specific language or extensive source-level changes, Laplace achieves all of these benefits while requiring no changes to existing binaries to track uncertainty through them. Compared to repeated Monte Carlo re-executions of applications on a conventional microarchitecture, Laplace achieves the same level of uncertainty tracking accuracy with 2,076× fewer executed instructions on average (up to 21,343× fewer).

Effective computational discretization scheme for nonlinear dynamical systems

Numerical methods are essential to investigate and apply nonlinear continuous-time dynamical systems in many fields of science and engineering and discretization schemes are required to obtain the solutions of such dynamical systems. Although computing power has been speedily growing in recent decades, embedded and large-scale problems have motivated significant research to improve the computational efficiency. Nevertheless, few studies have focused on finite precision limitation on discretization schemes due to round-off effects in floating-point number representation. In this paper, a computational effective discretization scheme for nonlinear dynamical systems is introduced. By means of a theorem, it is shown that high-order terms in the Runge-Kutta method can be neglected with no accuracy loss. The proposed approach is illustrated using three well-known systems, namely the Rössler systems, the Lorenz equations and the Sprott B system. The number of mathematical operations and simulation time have reduced up to 81.1% and 90.7%, respectively. Furthermore, as the step-size decreases, the number of neglected terms increases due to the precision of the computer. Yet, accuracy, observability of dynamical systems and the largest Lyapunov are preserved. The adapted scheme is effective, reliable and suitable for embedded and large-scale applications.

Efficient Floating Point Arithmetic for Quantum Computers

One of the major promises of quantum computing is the realization of SIMD (single instruction - multiple data) operations using the phenomenon of superposition. Since the dimension of the state space grows exponentially with the number of qubits, we can easily reach situations where we pay less than a single quantum gate per data point for data-processing instructions which would be rather expensive in classical computing. Formulating such instructions in terms of quantum gates, however, still remains a challenging task. Laying out the foundational functions for more advanced data-processing is therefore a subject of paramount importance for advancing the realm of quantum computing. In this paper, we introduce the formalism of encoding so called-semi-boolean polynomials. As it turns out, arithmetic Z/2^<i>n</i>Z ring operations can be formulated as semi-boolean polynomial evaluations, which allows convenient generation of unsigned integer arithmetic quantum circuits. For arithmetic evaluations, the resulting algorithm has been known as Fourier-arithmetic. We extend this type of algorithm with additional features, such as ancilla-free in-place multiplication and integer coefficient polynomial evaluation. Furthermore, we introduce a tailormade method for encoding signed integers succeeded by an encoding for arbitrary floating-point numbers. This representation of floating-point numbers and their processing can be applied to any quantum algorithm that performs unsigned modular integer arithmetic. We discuss some further performance enhancements of the semi boolean polynomial encoder and finally supply a complexity estimation. The application of our methods to a 32-bit unsigned integer multiplication demonstrated a 90% circuit depth reduction compared to carry-ripple approaches.

The floating point: Tales of the unexpected

Digital computation is central to almost all scientific endeavors and has become integral to university physics education. Students collect experimental data using digital devices, process data using spreadsheets and graphical software, and develop scientific programming skills for modeling, simulation, and computational work. Issues associated with the floating-point representation of numbers are rarely explored. In this article, problems of floating point are divided into three categories: significant-figure limits, propagation of floating-point representation error, and rounding. For each category, examples are presented of unexpected ways, in which the digital representation of floating-point numbers can impact the veracity of scientific results. These examples cover aspects of classical dynamics, numerical integration, cellular automata, statistical analysis, and digital timing. Suggestions are made for curriculum enhancement and project-style investigations that reinforce the issues covered at a level suitable for physics undergraduate students.

Accelerating Spike-by-Spike Neural Networks on FPGA With Hybrid Custom Floating-Point and Logarithmic Dot-Product Approximation

Spiking neural networks (SNNs) represent a promising alternative to conventional neural networks. In particular, the so-called Spike-by-Spike (SbS) neural networks provide exceptional noise robustness and reduced complexity. However, deep SbS networks require a memory footprint and a computational cost unsuitable for embedded applications. To address this problem, this work exploits the intrinsic error resilience of neural networks to improve performance and to reduce hardware complexity. More precisely, we design a vector dot-product hardware unit based on approximate computing with configurable quality using hybrid custom floating-point and logarithmic number representation. This approach reduces computational latency, memory footprint, and power dissipation while preserving inference accuracy. To demonstrate our approach, we address a design exploration flow using high-level synthesis and a Xilinx SoC-FPGA. The proposed design reduces 20.5× computational latency and 8× weight memory footprint, with less than 0.5% of accuracy degradation on a handwritten digit recognition task.

Random number generators produce collisions: Why, how many and more

It seems surprising that when applying widely used random number generators to generate one million random numbers on modern architectures, one obtains, on average, about 116 collisions. This article explains why, how to mathematically compute such a number, why they often cannot be obtained in a straightforward way, how to numerically compute them in a robust way and, among other things, what would need to be changed to bring this number below 1. The probability of at least one collision is also briefly addressed, which, as it turns out, again needs a careful numerical treatment. Overall, the article provides an introduction to the representation of floating-point numbers on a computer and corresponding implications in statistics and simulation. All computations are carried out in R and are reproducible with the texttt included in this article.

New Research Results of Algorithms for Analysis of Changing Electric Circuits

New research results in the field of accelerated analysis of linear and linearizable electrical circuits with variable elements are presented. The considered algorithms are based on a new modification of Gaussian elim-ination for solution of systems of linear equations. The researches was directed on a systematic thorough study of the phenomenon of quasi-stabilization of the error discovered by the authors during multiple repeated cir-cuit analyzes. The Sherman-Morrison algorithm and the iterative refinement algorithm are considered as alter-natives. To control the results of numerical experiments, the representation of floating-point numbers in the bi-nary128 format of the IEEE 754 standard is used. A method for the machine formation of mathematical models of circuits with an unlimited number of elements is proposed.

Using of Redundant Signed-Digit Numeral System for Accelerating and Improving the Accuracy of Computer Floating-Point Calculations

The article proposes a method for software implementation of floating-point computations on a graphics processing unit (GPU) with an increased accuracy, which eliminates sharp increase in rounding errors when performing arithmetic operations of addition, subtraction or multiplication with numbers that are significantly different from each other in magnitude. The method is based on the representation of floating-point numbers in the form of decimal fractions that have uniform distribution within a range and the use of redundant signed-digit numeral system to speed up calculations. The results of computational experiments for evaluating the effectiveness of the proposed approach are presented. The effect of accelerating computations is obtained for the problems of calculating the sum of an array of numbers and determining the dot product of vectors. The proposed approach is also applicable to the discrete Fourier transform.

Cause and origin of moire interferences in recursive processes and with fixed-point and floating-point data types

In the calculation of the map of periods of the Mandelbrot Set and working with fixed-point and floating-point real number representation systems, moire interference patterns always appear. These result from the discretization that every computer sets in the finite encoding of its real values. It is not known from which original layers such interferences are obtained and how these initial layers are generated. This article aims at answering these two questions. In order to search these original layers thousands of images —created by means of two different encodings of the real values and with different accuracies— of the map of periods were analyzed. Some points with constant location, called Source Points, around which regular patterns with hyperbolic configuration are generated, were located as well. This answers the questions regarding the cause and origin of moire interferences. Some characteristics of Source Points are described hereinafter. It is also justified why moire interferences in the plane of periods have a hypersensitive behavior. Finally, it is shown how in dynamic systems accuracy does not only affect the precision of results, but also the configuration of the results themselves. The following interesting reflection is, therefore, open to discussion: if the real world should be seen with a fractal way of looking, accuracy configurates then the essence of everything.

The Multiplication Method with Scaling the Result for High-Precision Residue Positional Interval Logarithmic Computations

Introduction. The solution of the simulation problems critical to rounding errors, including the problems of computational mathematics, mathematical physics, optimal control, biochemistry, quantum mechanics, mathematical programming and cryptography, requires the accuracy from 100 to 1 000 decimal digits and more. The main lack of high-precision software libraries is a significant decrease of the speed-in-action, unacceptable for practical problems, in particular, when performing multiplication. A way to increase computation performance over very long numbers is using the residue number system. In this work, we discuss a new fast multiplication method with scaling the result using original hybrid residue positional interval logarithmic floating-point number representation. Materials and Methods. The new way of the organizing numerical information is a residue positional interval logarithmic number representation in which the mantissa is presented in the residue number system, and information on an absolute value (the characteristic) in the interval logarithmic number system that makes it possible to accelerate performance of comparison and scaling is developed to increase the speed of calculations; to compare modular numbers, the provisions of interval analysis are used; to scale modular numbers, the properties of the logarithmic number system and approximate interval calculations using the Chinese reminder theorem are used. Results. A new fast multiplication method of floating-point residue-represented numbers is developed and patented; the authors evaluated the developed method speed-in action, compared the developed method with classical and pipelined multiplication methods of long numbers. Discussion and Conclusion. The developed method is 2.4–4.0 times faster than the pipelined multiplication method, and is 6.4–12.9 times faster than classical multiplication methods.

Prototyping of Nonlinear Time-Stepped Finite Element Simulation for Linear Induction Machines on Parallel Reconfigurable Hardware

The finite element method (FEM) is widely used for accurate design and analysis of electric machines; however, it suffers from long execution time. In this paper, for the first time hardware acceleration of two-dimensional FEM for a single-sided linear induction motor on the field programmable gate array (FPGA) is proposed. The nonlinearity of the iron core as well as the movement are taken into consideration. A new sparse solver is proposed based on left-looking Gilbert–Peierls algorithm for the system of linear equations of FEM that need to be solved in different iterations and time steps. Implementation of the model is performed in a massively paralleled and deeply pipelined hardware architecture using VHDL coding with single precision floating-point number representation. The proposed emulation was performed at various time steps resulting in significant average speedup of 9.73 times in comparison with JMAG-Designer as a commercial finite element software, and the overall hardware latency of each time step for the emulation was 49.2 ms in average with minimum achievable FPGA clock of 5.59 ns.

Implementation of neuro-fuzzy system with modified high performance genetic algorithm on embedded systems

In this paper implementation of ANFIS on embedded systems based on single-core and multi-core ARM processors is presented. A novel evolutionary optimization tool named, modified high performance genetic algorithm (mHPGA) with bacterial conjugation operator is applied to ANFIS as a training method. Fixed point and floating point number representations are applied and compared. Moreover new mutation algorithm has been proposed for fixed point numbers. The proposed method is designed to sweep numbers space to search possible solutions in large state space. Concurrency nature of mHPGA benefits implementation of multi threading feature on ARM cortex-A53 with four cores.

Machine-coded genetic operators and their performances in floating-point genetic algorithms

Machine-coded genetic algorithms (MCGAs) use the byte representation of floating-point numbers which are encoded in the computer memory. Use of the byte alphabet makes classical crossover operators directly applicable in the floating-point genetic algorithms. Since effect of the byte-based mutation operator depends on the location of the mutated byte, the byte-based mutation operator mimics the functionality of its binary counterpart. In this paper, we extend the MCGA by developing new type of byte-based genetic operators including a random mutation and a random dynamic mutation operator. We perform a simulation study to compare the performances of the byte-based operators with the classical FPGA operators using a set of test functions. The prepared software package, which is freely available for downloading, is used for the simulations. It is shown that the byte-based genetic search obtains precise results by carrying out the both exploration and exploitation tasks by discovering new fields of the search space and performing a local fine-tuning. It is also shown that the introduced byte-based operators improve the search capabilities of FPGAs by means of convergence rate and precision even if the decision variables are in larger domains.

Numerical Optimization Problems of Sine-Wave Fitting Algorithms in the Presence of Roundoff Errors

In this paper, the effect of roundoff errors on sine-wave fitting algorithms is investigated. It is shown that the standard calculation of sine-wave parameters may result in unexpectedly large errors, even with floating-point number representation. The three- and four-parameter least squares, the maximum likelihood, and the quantile-based estimator (QBE) methods are investigated. It is pointed out that imprecise phase storage and summation affect almost every sine-wave fitting algorithm. Furthermore, the necessary solution for sometimes ill-conditioned systems and the imprecisely evaluated distribution of the observation noise, as additional error sources, are also shown to influence a part of the fitting methods. Besides error descriptions, compensation techniques are also suggested in order to mitigate the effect of error sources. The enhancement of precision and robustness due to these suggestions is demonstrated, while keeping the given limited precision number representation platform. The QBE is shown to overcome roundoff error problems when its applicability conditions are fulfilled. In addition, its performance in comparison with that of the least squares estimator is highlighted. Finally, it is pointed out that the investigated methods show similar sensitivity to the inaccurate knowledge of the frequency of the sine wave.

HexEx

State-of-the-art hex meshing algorithms consist of three steps: Frame-field design, parametrization generation, and mesh extraction. However, while the first two steps are usually discussed in detail, the last step is often not well studied. In this paper, we fully concentrate on reliable mesh extraction. Parametrization methods employ computationally expensive countermeasures to avoid mapping input tetrahedra to degenerate or flipped tetrahedra in the parameter domain because such a parametrization does not define a proper hexahedral mesh. Nevertheless, there is no known technique that can guarantee the complete absence of such artifacts. We tackle this problem from the other side by developing a mesh extraction algorithm which is extremely robust against typical imperfections in the parametrization. First, a sanitization process cleans up numerical inconsistencies of the parameter values caused by limited precision solvers and floating-point number representation. On the sanitized parametrization, we extract vertices and so-called darts based on intersections of the integer grid with the parametric image of the tetrahedral mesh. The darts are reliably interconnected by tracing within the parametrization and thus define the topology of the hexahedral mesh. In a postprocessing step, we let certain pairs of darts cancel each other, counteracting the effect of flipped regions of the parametrization. With this strategy, our algorithm is able to robustly extract hexahedral meshes from imperfect parametrizations which previously would have been considered defective. The algorithm will be published as an open source library [Lyon et al. 2016].

Hardware-in-the-Loop Emulation of Linear Induction Motor Drive for MagLev Application

Linear induction machines are widely used in transportation systems due to their many advantages. Design and prototyping of electric machines are an expensive and time-consuming process; hardware-in-the-loop simulation provides an efficient alternative. In this paper, a field-programmable gate array-based real-time digital emulation of single-sided linear induction motor with the drive system is proposed. Implementation of the model is performed in both fixed-point using Xilinx system generator and floating-point number representations using a handwritten VHSIC Hardware Description Language code. Then, an evaluation in terms of real-time step-size and accuracy as well as hardware resource utilization is provided. The whole design was fully paralleled, which resulted in a considerable reduction of model execution time. The minimum time step of 2.3 and 0.8 $\mu \text{s}$ was achieved for floating-point and fixed-point implementations, respectively. The results of the real-time simulation are verified by the experimental results as well as a 2-D finite-element simulation in JMAG software.

Exploiting Binary Floating-Point Representations for Constraint Propagation

Floating-point computations are quickly finding their way in the design of safety- and mission-critical systems, despite the fact that designing floating-point algorithms is significantly more difficult than designing integer algorithms. For this reason, verification and validation of floating-point computations are hot research topics. An important verification technique, especially in some industrial sectors, is testing. However, generating test data for floating-point intensive programs proved to be a challenging problem. Existing approaches usually resort to random or search-based test data generation, but without symbolic reasoning it is almost impossible to generate test inputs that execute complex paths controlled by floating-point computations. Moreover, because constraint solvers over the reals or the rationals do not natively support the handling of rounding errors, the need arises for efficient constraint solvers over floating-point domains. In this paper, we present and fully justify improved algorithms for the propagation of arithmetic IEEE 754 binary floating-point constraints. The key point of these algorithms is a generalization of an idea by B. Marre and C. Michel that exploits a property of the representation of floating-point numbers.

Numerical Validation with the Discrete Stochastic Arithmetic on New Architectures

Now Scientists are facing up to a new age. Power of computer enables us to perform experiences in silicio . Modelisations are enough accurate to obtain results closed to reality, although they sometimes fail. Several reasons can be given: the modelisation is not close enough to reality, the model is unstable, etc. One reason is often forgotten: errors coming from computers. Indeed, numerical computations are based on floating point number representation which is only an approximation of real numbers. Each operation generates a very small error that can propagate and cause the computed result to become completely different than the expected one. Several methods are well known to study the round-off errors propagation. The talk will focus on the DSA (discrete stochastic arithmetic) and its implementation: the CADNA software. The DSA allows us to study the round-off error propagation in all codes written in Fortran, C or C++ with few code modifications. This method is powerful but a bottleneck is the increase of the over-cost in computation time. Explanations and a solution will be given. In a second part, an overview of the CADNA implementation on multicore, distributed systems and GPU will be developed. ...

DRAM-Based Statistics Counter Array Architecture With Performance Guarantee

The problem of efficiently maintaining a large number (say millions) of statistics counters that need to be updated at very high speeds (e.g., 40 Gb/s) has received considerable research attention in recent years. This problem arises in a variety of router management and data streaming applications where large arrays of counters are used to track various network statistics and implement various counting sketches. It proves too costly to store such large counter arrays entirely in SRAM, while DRAM is viewed as too slow for providing wirespeed updates at such high line rates. In particular, we propose a DRAM-based counter architecture that can effectively maintain wirespeed updates to large counter arrays. The proposed approach is based on the observation that modern commodity DRAM architectures, driven by aggressive performance roadmaps for consumer applications, such as video games, have advanced architecture features that can be exploited to make a DRAM-based solution practical. In particular, we propose a randomized DRAM architecture that can harness the performance of modern commodity DRAM offerings by interleaving counter updates to multiple memory banks. The proposed architecture makes use of a simple randomization scheme, a small cache, and small request queues to statistically guarantee a near-perfect load-balancing of counter updates to the DRAM banks. The statistical guarantee of the proposed randomized scheme is proven using a novel combination of convex ordering and large deviation theory. Our proposed counter scheme can support arbitrary increments and decrements at wirespeed, and they can support different number representations, including both integer and floating point number representations.

OpenGL ES 1.1 implementation based on OpenGL

In this paper, we present an efficient way of implementing OpenGL ES 1.1 3D graphics API library for the environments with hardware-supported OpenGL facility, typically as desktop PCs. Although OpenGL ES was started from the existing OpenGL features, it rapidly became the standard 3D graphics library customized for embedded systems through introducing fixed-point arithmetic operations, buffer management with fixed-point data type supports, completely new texture mapping functions and others. Currently, it is the official 3D graphics API for Google Android, Apple iPhone, Sony PlayStation3, etc. In this paper, we achieved improvements on the arithmetic operations for the fixed-point number representation, which is the most characteristic data type for OpenGL ES 1.1. For the conversion of fixed-point data types to the floating-point number representations for the underlying OpenGL, we show the way of efficient conversion processes even with satisfying OpenGL ES standard requirements. We also introduced a specialized memory management scheme to manage the converted data from the buffer containing fixed-point numbers. In the case of texture processing, the requirements in both standards are quite different, and thus we used completely new software-implementations. Our final implementation of OpenGL ES library provides all of more than 200 functions in the standard specification and passed its conformance test, to show its compliance with the standard. From the efficiency point of view, we measured its execution times for several OpenGL ES-specific application programs and achieved remarkable improvements.