Abstract
We present algorithms for performing the five elementary arithmetic operations ( <inline-formula><tex-math notation="LaTeX">$+$</tex-math></inline-formula> , <inline-formula><tex-math notation="LaTeX">$-$</tex-math></inline-formula> , ×, <inline-formula><tex-math notation="LaTeX">$\div$</tex-math></inline-formula> , and <inline-formula><tex-math notation="LaTeX">$\sqrt{\phantom{x}}$</tex-math></inline-formula> ) in floating point arithmetic with stochastic rounding, and demonstrate the value of these algorithms by discussing various applications where stochastic rounding is beneficial. The algorithms require that the hardware be compliant with the IEEE 754 floating-point standard and that a floating-point pseudorandom number generator be available. The goal of these techniques is to emulate stochastic rounding when the underlying hardware does not support this rounding mode, as is the case for most existing CPUs and GPUs. By simulating stochastic rounding in software, one has the possibility to explore the behavior of this rounding mode and develop new algorithms even without having access to hardware implementing stochastic rounding—once such hardware becomes available, it suffices to replace the proposed algorithms by calls to the corresponding hardware routines. When stochastically rounding double precision operations, the algorithms we propose are between 7.3 and 19 times faster than the implementations that use the GNU MPFR library to simulate extended precision. We test our algorithms on various tasks, including summation algorithms and solvers for ordinary differential equations, where stochastic rounding is expected to bring advantages.
Highlights
The IEEE 754-1985 standard for floating-point arithmetic specifies four rounding modes: the default round-to-nearest, which we denote by RN, and three directed rounding modes, round-toward-+∞, round-toward-−∞, and round-towardzero, which we denote by RU, RD, and RZ, respectively
Our goal is to show that recursive and compensated summation stagnate with the standard IEEE 754 rounding mode but not when stochastic rounding is used; stagnation is achieved in bfloat16 and binary16 arithmetics for k well below 105
Our algorithms for emulating stochastically rounded elementary arithmetic operations, along with the code for binary64 precision arithmetic that we provide, will allow those interested in looking into this problem to access arithmetic with stochastic rounding without requiring the use of MPFR or alternative multiple-precision libraries
Summary
The IEEE 754-1985 standard for floating-point arithmetic specifies four rounding modes: the default round-to-nearest, which we denote by RN, and three directed rounding modes, round-toward-+∞, round-toward-−∞, and round-towardzero, which we denote by RU, RD, and RZ, respectively. Monte Carlo arithmetic is used by tools such as Verificarlo [6] and Verrou [7] in order to estimate the impact of round-off errors in floating-point computation, but we are not aware of any examples of use in numerical software All these tools propose to run an a program multiple times using stochastic or Monte Carlo arithmetic and use the set of sampled results to draw conclusions on the propagation of rounding errors and the numerical stability of the same code when run with deterministic rounding. Stochastic rounding aims to capture more of the information stored in the least significant of the bits that are lost when rounding This benefit should be understood in a statistical sense: stochastic rounding may produce an error larger than that of roundto-nearest on a single rounding operation, but over a large number of roundings it may help to obtain a more accurate result, as errors of opposite signs cancel out.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
