Quantizer Output Research Articles

Robust memoryless quantization for minimum signal distortion

Robust quantizers are designed for situations where there is only an incomplete statistical description of the quantizer input. The goal of the design is to closely approximate quantizer inputs by quantizer outputs without using more than a specified number of quantization levels. The exact probability distribution of the input is unknown, but this distribution is known to belong to some set <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</tex> . The primary, set <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</tex> considered is the set of all unimodal probability distributions which satisfy generalized moment constraint (e.g., mean-square value less than or equal to a constant). A quantizer is derived which minimizes over all quantizers the maximum distortion over all distributions in <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</tex> . This robust quantizer guarantees a significanfiy lower worst case distortion than the classical Gaussian-optimal quantizer, while performing nearly as well as the Gaussian-optimal quantizer when the input is, in fact, Gaussian.

Asymptotic quantization error of continuous signals and the quantization dimension

Extensions of the limiting qnanfizafion error formula of Bennet are proved. These are of the form <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{s,k}(N,F)=N^{-\beta}B</tex> , where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> is the number of output levels, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">D_{s,k}(N,F)</tex> is the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">s</tex> th moment of the metric distance between quantizer input and output, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">\beta,B&gt;0,k=s/\beta</tex> is the signal space dimension, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</tex> is the signal distribution. If a suitably well-behaved <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> -dimensional signal density <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f(x)</tex> exists, <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B=b_{s,k}[\int f^{\rho}(x)dx]^{1/ \rho},\rho=k/(s+k)</tex> , and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b_{s,k}</tex> does not depend on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> . For <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k=1,s=2</tex> this reduces to Bennett's formula. If <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</tex> is the Cantor distribution on <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[0,1],0&lt;k=s/ \beta=\log 2/ \log 3&lt;1</tex> and this <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</tex> equals the fractal dimension of the Cantor set <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">[12,13]</tex> . Random quantization, optimal quantization in the presence of an output information constraint, and quantization noise in high dimensional spaces are also investigated.

Polar Quantization of a Complex Gaussian Random Variable

We solve numerically the optimum fixed-level non-uniform and uniform quantization of a circularly symmetric complex (or bivariate) Gaussian random variable for the mean absolute squared error criterion. For a given number of total levels, we determine its factorization into the product of numbers of magnitude and phase levels that produces the minimum distortion. We tabulate the results for numbers of "useful" output levels up to 1024, giving their optimal factorizations, minimum distortion, and entropy. For uncoded quantizer outputs, we find that the optimal splitting of rate between magnitude and phase, averaging to 1.52 and 1.47 bits more in the phase angle than magnitude for optimum and uniform quantization, respectively, compares well with the optimal polar coding formula Of 1.376 bits of Pearlman and Gray [3]. We also compare the performance of polar to rectangular quantization by real and imaginary parts for both uncoded and coded output levels. We find that, for coded outputs, both polar quantizers are outperformed by the rectangular ones, whose distortion-rate curves nearly coincide with Pearlman and Gray's polar coding bound. For uncoded outputs, however, we determine that the polar quantizers surpass in performance their rectangular counterparts for all useful rates above 6.0 bits for both optimum and uniform quantization. Below this rate, the respective polar quantizers are either slightly inferior or comparable.

Optimal Quantization of the Rayleigh Probability Distribution

We have derived an efficient and general algorithm for optimal quantization and applied it to the important, but neglected Rayleigh random variable. The optimal quantization range and output levels for the distortion criterion of minimum mean squared error have been calculated and tabulated for numbers of output levels from 1 to 64. We also compute and report the corresponding distortions and output entropies. For the sake of comparison, we also calculate and tabulate level spacings, minimum errors, and entropies for the optimal uniform (equal step size) quantization of the Rayleigh density. Distortion-rate curves are presented for uncoded and entropy coded quantizer outputs and compared with the distortion-rate function. We judge that these results will be useful to workers in the fields of optical holography, image processing, and communications.

Shifted band beamformer: Theory and concept

This paper introduces a fundamentally different beamformer structure which can be used to process bandpass sensor signals.The beamformer operates directly on frequency translated versions of the input signals to compute a frequency translated version of the beam output. This new technique, which is referred to as the shifted band beamformer, is functionally a time domain beamformer but it combines attributes of both time and frequency domain beamforming. It is especially compatible with systems where the sensor data is initially basebanded to facilitate the use of bandwidth sampling procedures. Beamformer vernier delay and throughput requirements are relaxed since the delay quantization and beam output rate depends on the highest frequency component of the basebanded signal rather than that of the bandpass signal. The shifted band beamformer can also use interpolation [R. G. Pridham and R. A. Mucci, “A Novel Approach to Digital Beamforming,” J. Acoust. Soc. Am. 63, 425–434 (1978)] to achieve further hardware savings. Relationship between the new structure and frequency domain beamforming is also discussed.

Quantization Based on the Mean-Absolute-Error Criterion

Performance criteria for the design of optimum quantizers are considered. A distance criterion for quantizer input and output probability distribution functions is formulated, and its relationship to the usual distortion criteria is established. The criterion based on absolute error is shown to have unique properties, justifying its use as a performance index for optimum quantization. Numerical results are presented, and the implication for adaptive quantization of the use of this criterion is discussed.

Subjective Evaluation of an Adaptive Differential Voice Encoder with Oversampling and Entropy Coding

An adaptive differential speech encoder was assessed using subjective evaluation procedures. The coder's adaptive quantizer was similar to the one used by Cohn and Melsa [1] and the predictor involved nonadaptive previous-sample feedback. The digital channel used to transmit the quantizer output levels was assumed error-free. Paired comparison tests were used to obtain scaled isopreference contours on the <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N-f</tex> plane, where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> denote, respectively, the number of quantizer output levels and the sampling rate relative to the Nyquist rate. These contours were used to determine the subjective signal-to-noise ratio vs. <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> , maximum subjective signal-signal-to-noise ratios vs. bit rate, optimum values of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">N</tex> , and bit-rate savings which occur when entropy coding is used instead of natural coding of the quantizer output levels. Entropy coding yielded a bit rate approximately equal to threequarters that for natural coding and Nyquist-rate sampling minimized the bit rate in each case. Savings of from one to two bits occurred when ADPCM was compared with nonadaptive DPCM. The fact that our system was better than others for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f \simeq 1.0</tex> but worse for <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f \gsim 2.0</tex> indicates the need to modify our quantizer adaptation algorithm as the sampling rate increases relative to the Nyqusit rate.

Adaptive Companding of Picture Signals in a Predictive Coder

This paper describes a technique for varying the quantization characteristics in a Differential Pulse Code Modulation (DPCM) System. A multiplier, based on already transmitted picture elements (pels) which are spatially close to the pel being coded, is used to multiply the prediction error in a DPCM system. This multiplied prediction error is quantized by a fixed quantizer. Coarseness of quantization is thus dependent on the value of the multiplier, which is adjusted on the basis of the sensitivity of the human observers to quantization errors. The implementation is particularly useful in those cases where the quantizer outputs are coded using fixed length code words. Computer simulations on a variety of pictures indicate that a good quality picture can be produced by using a quantizer having between 8 to 10 levels and a two-dimensional predictor. Similar picture quality can be obtained by a simple nonadaptive previous element DPCM system having a quantizer with 13 to 16 levels.

The Autocorrelation Function of the Output of an N-Level Quantizer with an Angle-Modulated Input Signal

A readily computable formula is given for the autocorrelation function of the output of an arbitrary time-invariant nonlinearity with an arbitrary angle-modulated input. The general result is specialized to the case of an odd-symmetric, but otherwise arbitrary, quantizer. It is shown that to evaluate the output autocorrelation the second-order characteristic function (SOCF) of the modulation is required. Using previously derived SOCFs for a constant, a Gaussian random process, and a generalized random telegraph wave (RTW), the output autocorrelation is given explicitly. Using the SOCF for a binary phase-shift keyed (BPSK) signal, derived here for the first time, the output autocorrelation with BPSK modulation is also presented. Finally, the autocorrelation functions of the modulated input processes are given by reducing the quantizer to a linearity, with the results for the generalized RTW and BPSK modulations being new.

On quantizers for DPCM coding of picture signals

A measure of picture quality for simple element, differentially coded pictures is developed based on certain subjective tests. The measure weights the quantization noise according to its visibility. It is shown that the measure correlates well with the picture quality determined on a standard impairment scale. Optimization of DPCM quantizers is done for this and for the mean-square measure of picture quality. Performance of the following types of quantizers is evaluated in terms of entropy of the quantized output and the picture quality: a) minimum mean-square error quantizers with a fixed number of levels, b) minimum mean-square error quantizers with fixed entropy, c) minimum mean-square subjective distortion quantizers with a fixed number of levels, d) minimum mean-square subjective distortion quantizers with fixed entropy, and e) uniform quantizers. It is concluded that for a fixed number of levels and a fixed word-length coding of the quantizer outputs, the quantizers in c) outperform those in a); and with variable length coding, the quantizers in d) perform better than all of the other quantizers having the same entropy. The sensitivity of the approach to variation of picture content is also investigated.

Digital processing of Doppler sonar signals

The signal processing gain by digital beamforming in Doppler sonar was investigated in terms of the enhancement of signal-to-reverberation ratio (SRR) at the postbeamforming Doppler-filter output with an objective to determine the minimum number of bits necessary for the quantization of beamformer channel input signals without appreciable degradation of sonar performance. The formulation of the problem follows closely the author’s earlier work which extends Davenport’s analysis of bandpass limiter to the more general multichannel multibit signal processors. Owing to the Doppler shift of target signal frequency and the relatively narrow bandwidth of Doppler filters, we express the frequency selective effect in the nonlinear interacting power series in terms of a new set of coefficients cmk, which bear crucial significance in sonar performance. Extensive numerical calculations of ’’Doppler processing gain’’ were carried out to demonstrate that four-bit quantization of array transducer signals is theoretically sufficient to yield a performance essentially identical to that of a linear beamformer, but severe loss of gain could result if fewer than four bits are taken at the quantizer output. It is further demonstrated numerically and substantiated analytically that, for small input SRR, the processing gain of a digital beamformer with Doppler filters cannot exceed that of a linear (or optimized four-bit digital) beamformer even it its channel signals are quantized into more than four bits.

A Comparative Study of Various Quantization Schemes for Speech Encoding

In this paper, the performance limits, as given by the signal-to-noise ratio (s/n), are described for different speech-encoding schemes including adaptive quantization and (linear) adaptive prediction schemes. The comparison is made on the basis of computer simulations using 8-kHz-sampled speech signals of one speaker. Different bit rates (two bits per sample–five bits per sample) have been used. A three-bit-per-sample pcm scheme with a nonadaptive μ100 quantizer leads to an s/n value of approximately 9 dB. A maximum s/n value of approximately 25 dB has been reached using an encoding scheme including both adaptive quantization and adaptive prediction. Entropy coding of the quantizer output symbols leads to an additional gain in s/n of nearly 3 dB.

Quantization Error in Predictive Coders

Predictive coders have been suggested for use as analog data compression devices. Exact expressions for reconstructed signal error have been rare in the literature. In fact most results reported in the literature are based on the assumption of Gaussian statistics for prediction error. Predictive coding of first-order Gaussian Markov sequences are considered in this paper. A numerical iteration technique is used to solve for the prediction error statistics expressed as an infinite series in terms of Hermite polynomials. Several interesting properties of predictive coding are thereby demonstrated. First, prediction error is in fact close to Gaussian, even for the binary quantizer. Sencond, quantizer levels may be optimized at each iteration according to the calculated density. Finally, the existence of correlation between successive quantizer outputs is shown. Using the series solutions described above, performance in terms of meansquare reconstruction error versus bit rate can be shown to parallel the theoretical rate distortion function for the first-order Markov process by about 0.6 bits/sample at low bit rates.

New Results From a Mathematical Study of an Adaptive Quantizer

We consider a general class of multibit adaptive quantizers in which the quantizer function is modified at every sampling instant according to a recursive law with the transitions depending on the value of the quantizer output. We obtain a rather comprehensive set of basic properties of the device which explain the interrelationship of different aspects of the device behavior and their dependence on the parameters of the adaptation algorithm. For the quantitative analysis of the device, we give formulas and bounds for the mean time required for the quantizer function to adapt from an arbitrary initial state to the optimal. A feature new with this work is a unified treatment and a common body of results for quantizers with both bounded and unbounded range. This paper extends all the analytical results reported in an earlier paper, which dealt with a restricted class of quantizers having only four levels. We also present new results from a computational investigation on quantizers up to four bits (sixteen levels). These results indicate, for well-designed examples of the respective classes, the kinds of improvement in performance that can be expected in going from three-bit (eight-level) to four-bit quantizers and from uniform to nonuniform quantizers.

Signal-processing gain by digital beamforming in Doppler sonar

The signal-processing gain by digital beamforming in Doppler sonar was investigated in terms of the enhancement of signal-to-reverberation ratio (SRR) at the post-beamforming Doppler filter output with an objective to determine the minimum number of bits necessary for the quantization of beamformer channel input signals without appreciable degradation of sonar performance. It is shown that, in the present case, the simplified method of counting spectral lobes originally due to Davenport [W. B. Davenport Jr., J. Appl. Phys. 24, 720 (1953); for application to multichannel multibit signal processors, see H. S. C. Wang, J. Acoust. Soc. Am. 53, 929 (1973)] can only be applied with certain modifications in the evaluation of various signal and reverberation interacting power terms. Extensive numerical calculations of “Doppler processing gain” were then carried out to demonstrate that four-bit quantization of array transducer signals is theoretically sufficient to yield a performance essentially identical to that of a linear beamformer, but severe loss of gain could result if fewer than four bits are taken at the quantizer output. It is further demonstrated numerically and substantiated analytically that, for small input SRR, the processing gain of a digital beamformer with Doppler filters cannot exceed that of a linear (or optimized four-bit digital) beamformer even if its channel signals are quantized into more than four bits.

Theory of an Adaptive Quantizer

In an adaptive quantizer that has been used for speech encoding, the entire amplitude range expands or contracts by a multiplicative constant after each input sample. The constant M i depends only on the magnitude of the current quantizer output. Assuming independent identically distributed input samples, we show that the sequence of quantizer ranges is a stochastically stable process. Furthermore, we derive a key design equation, \Sigma p_{i}x logM_{i} = O , where p_{i}(x) is the probability that an input sample is in the i th magnitude interval when the ratio of quantizer range to rms signal level is x . A designer may specify x and solve this equation for multipliers that provide the desired steady-state performance. There are many such sets of multipliers and we show that the adaptation time constant associated with each set decreases as the ratio of the largest multiplier to the smallest multiplier is increased. On the other hand, the spread of the steady-state range distribution about the operating point can be made as small as desired by making this ratio sufficiently small. A bound is obtained for the tradeoff between responsiveness to changing input level and steady-state range accuracy.

Adaptive Quantization in Differential PCM Coding of Speech

We describe an adaptive differential PCM (ADPCM) coder which makes instantaneous exponential changes of quantizer step-size. The coder includes a simple first-order predictor and a time-invariant, minimally complex adaptation strategy. Step-size multipliers depend only on the most recent quantizer output, and input signals of unknown variance can be accommodated. We derive appropriate multiplier values from computer simulations with speech signals and with Gauss-Markov inputs. We compare performance of the ADPCM coder with conventional log-PCM, using both objective and subjective criteria. Finally, we describe an economical integrated hardware implementation of the ADPCM coder. We believe that at bit rates of 24 to 32 kb/s, ADPCM provides a robust and efficient technique for speech communication and for digital storage of speech.

The Application of Dither to the Quantization of Speech Signals

By adding a pseudorandom “dither” noise to a signal X that is to be quantized, and by subtracting an identical noise sequence from the quantizer output, it is possible to break up undesirable signal-dependent patterns in the quantization error sequence, without increasing the variance of the error E. The idea has been widely discussed in the context of picture coding, and it is the purpose of this talk to demonstrate application of the technique to the quantization of speech signals. Computer simulations have shown how the use of dither whitens the quantization error sequence in PCM encoding, and renders it more acceptable than signal-correlated errors of equal variance. We demonstrate, for conditions of dither and no dither, typical speech recordings, illustrative error waveforms, and data on signal-to-error correlation C; and indicate how the advantage of dithering increases monotonically with crudeness of signal quantization and becomes significant when the number of bits per sample is less than about six. An experimental evaluation of the perceptual effects of dither on speech has shown (a) strong preferences for dithered speech over straight PCM encoding at identical bit rates; (b) an increase in word intelligibility for dithered speech over PCM speech when 4–6 bits/sample were used; and (c) a decrease in word intelligibility for dithered speech over PCM speech when 2–3 bits/sample were used. The above results imply that dithering is most advantageous for speech quantized in the 4–6 bits/sample range.

Optimum quantizers and permutation codes

Amplitude quantization and permutation encoding are two of the many approaches to efficient digitization of analog data. It is shown in this paper that these seemingly different approaches actually are equivalent in the sense that their optimum rate versus distortion performances are identical. Although this equivalence becomes exact only when the quantizer output is perfectly entropy coded and the permutation code block length is infinite, it nonetheless has practical consequences both for quantization and for permutation encoding. In particular, this equivalence permits us to deduce that permutation codes provide a readily implementable block-coding alternative to buffer-instrumented variable-length codes. Moreover, the abundance of methods in the literature for optimizing quantizers with respect to various criteria can be translated directly into algorithms for generating source permutation codes that are optimum for the same purposes. The optimum performance attainable with quantizers (hence, permutation codes) of a fixed entropy rate is explored too. The investigation reveals that quantizers with uniformly spaced thresholds are quasi-optimum with considerable generality, and are truly optimum in the mean-squared sense for data having either an exponential or a Laplacian distribution. An attempt is made to provide some analytical insight into why simple uniform quantization is so good so generally.

Perceptual Evaluation of the Effects of Dither on Low Bit Rate PCM Systems

It has previously been shown that by adding a pseudo-random “dither” noise to a signal to be quantized, and by subtracting an identical noise sequence from the quantizer output, it is possible to break up undesirable signal-dependent patterns in the quantization error sequence without increasing the variance of the error. The effect of the dither noise becomes significant when the number of bits per sample is less than about six. An experimental evaluation of the perceptual effects of dither on speech has shown: (i) strong preferences for dithered speech over straight PCM encoding at identical bit rates, (ii) for low bit rates (2–4 bits/sample), a preference for dithered speech over PCM encoded speech even when the PCM speech had one more bit per sample than the dithered speech, (iii) an increase in word intelligibility for dithered speech over PCM speech when 4 to 6 bits/sample were used, (iv) a decrease in word intelligibility for dithered speech over PCM speech when 2 to 3 bits/sample were used.