Behavior of the Quantization Operator for Bandlimited, Nonoversampled Signals

Abstract

The process of quantization generates a loss of information, and, thus, the original signal cannot be reconstructed exactly from the quantized samples in general. However, it is desirable to keep the error as small as possible. In this paper, the quantization error is quantified in terms of several distortion measures. All these measures employ the difference between the original signal and the reconstructed signal, which is obtained by bandlimited interpolation of the quantized samples. We assume that the signals are bandlimited and that the samples are taken at Nyquist rate. It is shown that for signals in the Paley-Wiener space PW ? 1, the supremum of the reconstructed signal, and, hence, the quantization error cannot be bounded in the sense that there exists a bounded subset of PW ? 1 on which both quantities can increase unboundedly. This unexpected behavior is due to the nonlinearity of the quantization operator and the slow decay of the sinc function. The nonlinearity is essential for this behavior because every linear operator that fulfills a certain property of the quantization operator would otherwise have to be bounded. Furthermore, it is proven that for a fixed signal the possible quantization error increases as the quantization step size tends to zero. The treatment of the quantization error in this paper is completely deterministic.