On the variance of a centered random value roundoff error

Abstract

We derive two expressions for roundoff error variance, one for a rounded off random value with a zero mean and a given variance under uniform distribution and another for such a value under a normal. Also, an expression for truncation error variance for values under uniform distribution is obtained. An application of the expressions to analysis of processing essentially quantized data by a nonrecursive smoothing filter is shown. Also their applications to quantization error (quantization noise) analysis of general linear processing of quantized signals under uniform and normal distributions and to quantization error analysis of essentially quantized discrete transforms like DFT (discrete Fourier transform), DCT (discrete cosine transform), DWT (discrete Walsh transforms), wavelet transforms, and so on, to image, sound (audio), video and to general signal processing in many cases can be considered as useful. The effect of accuracy of using these expressions is the more, the more used quantization level and the less maximal signal amplitudes.