Floating-point Round-off Error Research Articles

Program Reconditioning: Avoiding Undefined Behaviour When Finding and Reducing Compiler Bugs

We introduce program reconditioning, a method for allowing program generation and differential testing to be used to find miscompilation bugs, and test-case reduction to be used to simplify bug-triggering programs, even when (a) the programming language of interest features undefined behaviour (UB) and (b) no tools exist to detect and avoid this UB. We present two program generation tools based on our reconditioning idea: GLSLsmith for the OpenGL Shading Language (GLSL), a widely-used language for graphics programming, and WGSLsmith for the WebGPU Shading Language (WGSL), a new language for web-based graphics rendering. GLSL features many UBs, but unlike for languages such as C and C++ no tools exist to detect them automatically. While the WGSL language specification features very limited UB, early WGSL implementations do exhibit UB, for reasons of initial implementation simplicity, making it challenging to test them to quickly detect and eliminate unrelated miscompilation bugs. Thanks to reconditioning, we show that GLSLsmith and WGSLsmith allow differential testing and test-case reduction to be applied to compilers for GLSL and WGSL for the first time, despite the unavailability of UB detection techniques for these languages. Through a large testing campaign, we have found 24 and 33 bugs in GLSL and WGSL compilers, respectively. We present experiments showing that when reconditioning is disabled, compiler testing leads to a high rate of test programs that appear to trigger miscompilation bugs, but actually just feature UB. We also present a novel approach to managing floating-point roundoff error using reconditioning, implemented for both GLSL and WGSL.

Quantization noise analysis in FBMC receivers and its effect on the BER performance

This paper analyzes the fixed-point and floating-point roundoff errors in different Filter Bank MultiCarrier (FBMC) receiver implementations. Two different structures are considered: the polyphase network based and frequency spreading based. Theoretical closed-form formulas for the total Quantization Noise Power (QNP) are derived and verified through simulation. The results show the effects of each parameter and signal processing block on the overall QNP. Also, the results show that the total QNP for an arbitrary set of parameters can be determined using the derived results without performing exhaustive simulations. The effects of the chosen bit-resolution on the bit-error rate performance are also analyzed. Furthermore – based on the theoretical results – FBMC receivers can be designed with an optimal number of bits used in the arithmetic units without significantly affecting the receiver performance.

A Safe Computational Framework for Integer Programming Applied to Chvátal’s Conjecture

We describe a general and safe computational framework that provides integer programming results with the degree of certainty that is required for machine-assisted proofs of mathematical theorems. At its core, the framework relies on a rational branch-and-bound certificate produced by an exact integer programming solver, SCIP, in order to circumvent floating-point round-off errors present in most state-of-the-art solvers for mixed-integer programs. The resulting certificates are self-contained and checker software exists that can verify their correctness independently of the integer programming solver used to produce the certificate. This acts as a safeguard against programming errors that may be present in complex solver software. The viability of this approach is tested by applying it to finite cases of Chvátal’s conjecture, a long-standing open question in extremal combinatorics. We take particular care to verify also the correctness of the input for this specific problem, using the Coq formal proof assistant. As a result, we are able to provide the first machine-assisted proof that Chvátal’s conjecture holds for all downsets whose union of sets contains seven elements or less.

Rigorous Estimation of Floating-Point Round-Off Errors with Symbolic Taylor Expansions

Rigorous estimation of maximum floating-point round-off errors is an important capability central to many formal verification tools. Unfortunately, available techniques for this task often provide very pessimistic overestimates, causing unnecessary verification failure. We have developed a new approach called Symbolic Taylor Expansions that avoids these problems, and implemented a new tool called FPTaylor embodying this approach. Key to our approach is the use of rigorous global optimization, instead of the more familiar interval arithmetic, affine arithmetic, and/or SMT solvers. FPTaylor emits per-instance analysis certificates in the form of HOL Light proofs that can be machine checked. In this article, we present the basic ideas behind Symbolic Taylor Expansions in detail. We also survey as well as thoroughly evaluate six tool families, namely, Gappa (two tool options studied), Fluctuat, PRECiSA, Real2Float, Rosa, and FPTaylor (two tool options studied) on 24 examples, running on the same machine, and taking care to find the best options for running each of these tools. This study demonstrates that FPTaylor estimates round-off errors within much tighter bounds compared to other tools on a significant number of case studies. We also release FPTaylor along with our benchmarks, thus contributing to future studies and tool development in this area.

Understanding the many-body expansion for large systems. II. Accuracy considerations.

To complement our study of the role of finite precision in electronic structure calculations based on a truncated many-body expansion (MBE, or "n-body expansion"), we examine the accuracy of such methods in the present work. Accuracy may be defined either with respect to a supersystem calculation computed at the same level of theory as the n-body calculations, or alternatively with respect to high-quality benchmarks. Both metrics are considered here. In applications to a sequence of water clusters, (H2O)N=6-55 described at the B3LYP/cc-pVDZ level, we obtain mean absolute errors (MAEs) per H2O monomer of ∼1.0 kcal/mol for two-body expansions, where the benchmark is a B3LYP/cc-pVDZ calculation on the entire cluster. Three- and four-body expansions exhibit MAEs of 0.5 and 0.1 kcal/mol/monomer, respectively, without resort to charge embedding. A generalized many-body expansion truncated at two-body terms [GMBE(2)], using 3-4 H2O molecules per fragment, outperforms all of these methods and affords a MAE of ∼0.02 kcal/mol/monomer, also without charge embedding. GMBE(2) requires significantly fewer (although somewhat larger) subsystem calculations as compared to MBE(4), reducing problems associated with floating-point roundoff errors. When compared to high-quality benchmarks, we find that error cancellation often plays a critical role in the success of MBE(n) calculations, even at the four-body level, as basis-set superposition error can compensate for higher-order polarization interactions. A many-body counterpoise correction is introduced for the GMBE, and its two-body truncation [GMBCP(2)] is found to afford good results without error cancellation. Together with a method such as ωB97X-V/aug-cc-pVTZ that can describe both covalent and non-covalent interactions, the GMBE(2)+GMBCP(2) approach provides an accurate, stable, and tractable approach for large systems.

Whfast: a fast and unbiased implementation of a symplectic Wisdom–Holman integrator for long-term gravitational simulations

We present WHFast, a fast and accurate implementation of a Wisdom-Holman symplectic integrator for long-term orbit integrations of planetary systems. WHFast is significantly faster and conserves energy better than all other Wisdom-Holman integrators tested. We achieve this by significantly improving the Kepler-solver and ensuring numerical stability of coordinate transformations to and from Jacobi coordinates. These refinements allow us to remove the linear secular trend in the energy error that is present in other implementations. For small enough timesteps we achieve Brouwer's law, i.e. the energy error is dominated by an unbiased random walk due to floating-point round-off errors. We implement symplectic correctors up to order eleven that significantly reduce the energy error. We also implement a symplectic tangent map for the variational equations. This allows us to efficiently calculate two widely used chaos indicators the Lyapunov characteristic number (LCN) and the Mean Exponential Growth factor of Nearby Orbits (MEGNO). WHFast is freely available as a flexible C package, as a shared library, and as an easy-to-use python module.

Lattice Boltzmann magnetohydrodynamics with current-dependent resistivity

Lattice Boltzmann magnetohydrodynamics is extended to allow the resistivity to be a prescribed function of the local current density. Current-dependent resistivities are used to model the so-called anomalous resistivity caused by unresolved small-scale processes, such as current-driven plasma microturbulence, that are excluded by the magnetohydrodynamics approximation. These models closely resemble the Smagorinsky eddy viscosity model used in large eddy simulations of hydrodynamic turbulence. Lattice Boltzmann implementations of the Smagorinsky model adjust the collision time in proportion to the local rate of strain, as obtained from the non-equilibrium parts of the hydrodynamic distribution functions. This works successfully even with a single relaxation time collision operator. However, the existing lattice Boltzmann magnetohydrodynamic implementation contains a spurious term in the evolution equation for the magnetic field that violates the divergence-free condition when the relaxation time varies in space. A correct implementation requires a matrix collision operator for the magnetic distribution functions. The relaxation time imposed on the antisymmetric component of the electric field tensor is calculated locally from the non-equilibrium part of the magnetic distribution functions, which determine the current, while the symmetric component remains subject to a uniform relaxation time to suppress the spurious term. The resulting numerical solutions are shown to converge to independent spectral solutions of the magnetohydrodynamic equations, and to preserve the divergence-free condition up to floating point round-off error.

Covariance Calculation for Floating-Point State-Space Realizations

This paper provides a new method for analyzing floating-point roundoff error for digital filters by using finite signal-to-noise models whose noise sources have variances proportional to the variance or power of the corrupted signals. With this model, a new expression for output error covariance of floating-point arithmetic is derived the in case of double or extended precision accumulation. The output error covariance shows that the optimal state space realization for floating point is the same as that of the fixed-point case, except for two cases: when the filter has poles extremely close to the unit circle or when final quantization to precisions shorter than single precision is employed. An explicit formula is found for determining the minimum number of mantissa bits for stable realization.

COVARIANCE CALCULATION FOR FLOATING POINT STATE SPACE REALIZATIONS

This paper provides a new method for analyzing floating point roundoff error for digital filters by using FSN (Finite Signal Noise) models whose noise sources have variances proportional to the the variance of the corrupted signals. With this model the output error covariance of floating point arithmetic is derived and it is shown that optimal state space realization can be different from the optimal structure of the fixed point case.

A note on geometric ergodicity and floating-point roundoff error

We consider the extent to which Markov chain convergence properties are affected by the presence of computer floating-point roundoff error. Both geometric ergodicity and polynomial ergodicity are considered. This paper extends previous work of Roberts et al. (J. Appl. Probab. 35 (1998) 1) to the case of proportional errors.

Thick-Restart Lanczos Method for Large Symmetric Eigenvalue Problems

In this paper, we propose a restarted variant of the Lanczos method for symmetric eigenvalue problems named the thick-restart Lanczos method. This new variant is able to retain an arbitrary number of Ritz vectors from the previous iterations with a minimal restarting cost. Since it restarts with Ritz vectors, it is simpler than similar methods, such as the implicitly restarted Lanczos method. We carefully examine the effects of the floating-point round-off errors on stability of the new algorithm and present an implementation of the partial reorthogonalization scheme that guarantees accurate Ritz values with a minimal amount of reorthogonalization. We also show a number of heuristics on deciding which Ritz pairs to save during restart in order to maximize the overall performance of the thick-restart Lanczos method.

Roundoff errors in block-floating-point systems

Block-floating-point representation is a special case of floating-point representation, where several numbers have a joint exponent term. In this paper, roundoff errors in signal processing systems utilizing block-floating-point representation are studied. Special emphasis is on analysis of quantization errors when data is quantized to a block-floating-point format and on analysis of roundoff errors in digital filters utilizing block-floating-point arithmetic, block-floating-point roundoff errors are found to depend on the signal level in the same way as floating-point roundoff errors, resulting in approximately constant signal-to-noise-ratios (SNRs) over relatively large dynamic range. Both the analysis and simulation results show that block-floating-point is an efficient number representation format. In data representation, a superior performance to fixed- or floating-point representations can be achieved with block-floating-point representation with same total number of bits per sample. In digital filters, block-floating-point arithmetic can provide comparable performance to floating-point arithmetic with reduced complexity.

Analysis of floating point roundoff errors in the estimation of higher-order statistics

A floating point roundoff error analysis in the estimation of higher-order statistics, moments or cumulants of real stationary processes from single data records is provided. Closed form expressions or upper bounds are derived for the mean and variance of the quantisation noise introduced in the estimation of the all-zero and all-T (diagonal slice) moments, power, skewness and kurtosis. Numerical and simulation results show that the roundoff noise can significantly affect the moment and cumulant estimates, especially when long data records are employed for the purpose of reducing the estimation variance. The obtained results can provide guidelines in choosing a processor with the appropriate register length (in number of bits) in applications that require the calculation of higher-order statistics.

Analysis of floating point roundoff errors using dummy multiplier coefficient sensitivities

A simple method for analyzing roundoff errors in floating-point digital filters is presented. The method is based on the coefficient sensitivities of dummy multipliers with gains of one and connects the roundoff error analysis to coefficient sensitivity analysis. A general formulation for calculating the roundoff noise level at the output of a digital filter is presented, and a simple upper bound for roundoff error accumulation is derived. The approach makes it possible to draw some general conclusions about the floating-point roundoff noise properties of certain filter structures. The cascade, parallel, and Gray-Markel lattice structures are analyzed in detail. The interaction between coefficient sensitivity and roundoff error is discussed, and several general conclusions are drawn. Numerical examples confirm that the method is accurate enough for practical use.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Floating point roundoff error in the prime factor FFT

The prime factor fast Fourier transform (PF FFT), developed by Kolba and Parks, makes use of recent computational complexity results by Winograd to compute the DFT with a fewer number of multiplications than that required by the FFT. Patterson and McClellan have derived an expression for the mean squared error (MSE) in the PF FFT, assuming finite precision fixed point arithmetic. In this paper, we derive an expression for the MSE in the PF FFT, assuming floating point arithmetic. This expression is quite complicated, so an upper bound on the MSE is also derived which is easier to compute. Simulation results are presented comparing the error in the PF FFT with both the derived bound and the error observed in a radix-2 FFT.