Lloyd-Max Quantization Research Articles

Optimal construction of subband coders using Lloyd-Max quantizers

A new method is presented for the analysis of the effects of Lloyd-Max quantization in subband filterbanks and for the optimal design of such filterbanks. A rigorous statistical model of a vector Lloyd-Max quantizer is established first, consisting of a linear time-invariant filter followed by additive noise uncorrelated/with the input. On the basis of this model, an expression for this variance of the error of a subband coder using Lloyd-Max quantizers is explicitly determined. Given analysis filters that statistically separate the subbands, it is shown that this variance is minimized if the synthesis filters are chosen, which mould achieve perfect reconstruction in lossless coding. The globally optimum of such a filterbank, minimizing the coder error variance, is further obtained by proper choice of its analysis filters. An alternative design method is also evaluated and optimized. In this, the errors correlated with the signal are set to zero, leaving a random error residue uncorrelated with the signal. This design method is optimized by choosing the analysis filters so as to minimize the random error variance. The results are evaluated experimentally in the realistic setting of a logarithmically split subband image coding scheme.

Optimal pyramidal decomposition for progressive multidimensional signal coding using optimal quantizers

Optimal hierarchical coding is sought, for progressive or scalable multidimensional signal transmission, by minimizing the variance of the error difference between the original image and its lower resolution renditions. The optimal, according to the above criterion, pyramidal coders are determined for images quantized using the optimal vector Lloyd-Max quantizers. A rigorous general statistical model of a vector Lloyd-Max quantizer is used, consisting of a linear time-invariant filter followed by additive noise uncorrelated with the input. Given arbitrary analysis filters, the optimal synthesis filters are found. The optimal analysis filters are subsequently determined, leading to formulas for globally optimal structures for pyramidal multidimensional signal decompositions. These structures produce replicas of the original image, which at lower resolutions retain as much similarity to the original as possible. This is highly useful for the progressive coding of two- or three-dimensional (2-D or 3-D) images needed in applications such as fast browsing through image databases. Furthermore, the minimization of the variance of the error image leads to minimization of the variance of the quantization noise for this image and, hence, to its optimally efficient compression. Experimental results illustrate the implementation and performance of the optimal pyramids in application for the coding of still 2-D images.

Adaptive scalar quantization without side information

In this paper, we introduce a novel technique for adaptive scalar quantization. Adaptivity is useful in applications, including image compression, where the statistics of the source are either not known a priori or will change over time. Our algorithm uses previously quantized samples to estimate the distribution of the source, and does not require that side information be sent in order to adapt to changing source statistics. Our quantization scheme is thus backward adaptive. We propose that an adaptive quantizer can be separated into two building blocks, namely, model estimation and quantizer design. The model estimation produces an estimate of the changing source probability density function, which is then used to redesign the quantizer using standard techniques. We introduce nonparametric estimation techniques that only assume smoothness of the input distribution. We discuss the various sources of error in our estimation and argue that, for a wide class of sources with a smooth probability density function (pdf), we provide a good approximation to a "universal" quantizer, with the approximation becoming better as the rate increases. We study the performance of our scheme and show how the loss due to adaptivity is minimal in typical scenarios. In particular, we provide examples and show how our technique can achieve signal-to-noise ratios within 0.05 dB of the optimal Lloyd-Max quantizer for a memoryless source, while achieving over 1.5 dB gain over a fixed quantizer for a bimodal source.

Perceptually optimum gray-scale transformation of multibin time history data

Gray-scale displays of a time history of data in multiple channels are encountered in various applications. The signal tracks in a noisy display often have to be detected by a human operator. To exploit the pattern-recognition capability of the human "receiver", the perceptual factors influencing visual discrimination needs to be incorporated into the gray-scale transformation scheme of the data. To this end, an objective function, viz. the total perceived entropy, is proposed in this paper for optimum gray-scale quantizer design. Two types of quantizers, one designed with unconstrained maximization of the objective function and another which maximizes the objective function with the constraint enforced by Weber's law, are proposed. It is demonstrated, using simulated spectrogram data, that these quantizers provide better signal track discrimination than Lloyd-Max quantization.

Pre/post-filter for performance improvement of transform coding

Abstract A technique for generating independent transform coefficients from any p -dependent signal has been developed. Since these coefficients are independent, the Lloyd-Max quantization efficiency is improved. In addition, these coefficients are shown to be Gaussian distributed. Therefore, the probability density function estimate is bypassed during the quantizer design. An all-pass (encryption) pre-filter is required by the proposed technique, and the filtered signal is obtained efficiently by an algorithm developed in this paper. An added benefit of this technique is the compatibility and higher security with respect to the conventional transform coding (TC), and the method is called the transform encryption coding (TEC). In addition, a post-filter is necessary for signal reconstruction. Due to the all-pass nature of the pre/post-filtering process, the mean-square quantization error for the complete process is equal to that for the intermediate independent Gaussian transform coefficients. Simulation results show TEC achieves about 1 dB coding gain compared with TC. Coded images without blocking effects are obtained at 0.5 bit/pixel (bpp) using TEC. The image quality is similar to that obtained by TC at 1 bpp. Even at 0.35 bpp, TEC performs in the same way as the recent ‘DCT/DST’ technique of Rose et al. (1990) and shows no blocking effects. Most significantly, TEC encoded image is insensitive and robust to the channel noise.

Subband coding systems incorporating quantizer models

A new method for dealing with the effects of quantization in a subband system is proposed. It uses the "gain plus additive noise" linear model for the Lloyd-Max quantizer. Based on this, it is demonstrated how, by an appropriate choice of synthesis filters, one can cancel all signal-dependent errors at the output of the system. The only remaining error is random in nature and not correlated with the input signal. We therefore have a tradeoff between the error being only random or having signal-dependent components as well (since the error variances in both cases are comparable). As a result of having only a random error, it is possible to reduce this error using, for example, a noise removal technique. The result is then extended to the case where the input is a multidimensional signal, and arbitrary sampling lattices are used, as well as to the QMF (alias cancellation) case. To demonstrate the validity of the proposed approach, two types of experiments on images are carried out: In a toy example, it is shown that using noise removal could be beneficial. For a more realistic coding scheme, however, it is demonstrated that even in the case when the model is no longer valid (when some of the subbands are discarded), the output error is still much less correlated with the input signal as opposed to the commonly used subband system, while visually, the reconstructed images look very similar.

A comparison of scalar quantization strategies for noisy channel data transmission

It is known that the combination of uniform quantization and entropy coding performs optimally at asymptotically high rates. However, due to synchronization problems resulting from the use of variable length codes, this is true only in the absence of transmission errors. Thus, the performance over noisy channels can be improved by using either fixed length codes and Lloyd-Max quantization or by adding end-of-line symbols. Computer simulations are used to compare the above two schemes for a variety of error rates, line lengths and amounts of error protection. It is found that the "uniform" scheme gives the best SNR for a given rate in the case of memoryless Gaussian and Laplacian sources, provided that its parameters are chosen sensibly. It is expected that these results will generalize to other sources.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Adaptive cosine transform coding for compression of Mars-96 ELISMA instrument data

Mars-96 ELISMA is a scientific project designed to measure the electric and magnetic fields around Mars when the Russian spacecraft Mars-96 orbits the planet. The data rate of the on-board ELISMA instruments will reach a figure which is about five times the telemetry rate allocated to them. To overcome this problem, a coding was designed to compress the data for the telemetry. This coding combines an adaptive cosine transform with marginal analysis. Data from the instruments are divided into blocks with each block converted by the cosine transform into coefficients. The marginal analysis method is performed on the coefficients to achieve a bit allocation vector specifying the level of the subsequent Lloyd-Max quantization of the coefficients. This paper describes the coding algorithm and also shows simulation results for the compression of geophysical data.

A structured fixed-rate vector quantizer derived from a variable-length scalar quantizer. II. Vector sources

For Pt.I see ibid., vol.39, no.3, p.851-67 (1993). The fixed-rate scalar-vector quantizer (SVQ) for quantizing stationary memoryless sources is extended to a specific type of vector source in which each component is a stationary memoryless scalar subsource independent of the other components. Algorithms for the design and implementation of the original SVQ are modified to apply to this case. The resulting SVQ, referred to as the extended SVQ (ESVQ), is then used to quantize stationary sources with memory (with known autocorrelation function). Numerical results are presented for the quantization of first-order Gauss-Markov sources using this scheme. It is shown that the ESVQ-based scheme performs very close to entropy-coded transform quantization while maintaining a fixed-rate output and outperforms the fixed-rate scheme that uses scalar Lloyd-Max quantization of the transform coefficients. It is also shown that this scheme performs better than implementable vector quantizers, especially at high rates. >

Adaptive channel error protection of subband encoded images

Protection of images that are encoded using subband coding from channel error is addressed. In this scheme the low-pass subband is encoded using DPCM (differential pulse-code modulation), and the other subbands are encoded using a scalar quantizer. The quantizers are all Lloyd-Max quantizers, from which the representation levels have fixed length codewords. First, considering only single errors in each codeword, a channel error distortion measure is derived for each quantizer, that is, for each subband. Codewords are assigned to the quantizer representation levels, yielding a low value of the distortion measure. Next, sets S/sub ij/ consisting of the jth bit from subband i are formed. Each set S/sub ij/ is assigned a particular BCH code C/sub ij/. An algorithm that optimally assigns BCH codes C/sub ij/ to each set S/sub ij/, based on a channel error distortion measure for the entire image, is derived. The protection scheme is adaptive, because each set of bits within each subband can be assigned a different error protection code. Examples show that this approach is preferable to assigning equal error protection codes to each set of bits. It is shown that in the case of a channel error probability of 10/sup -3/, only 5% to 10% extra bits are needed for adequate channel error protection.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Recursively indexed quantization of memoryless sources

A recursively indexed scalar quantizer that performs as well as high-dimensional vector quantizers for several important sources, without the attendant complexity, is presented. The recursively indexed quantizer provides a simple technique, both in terms of design and operation, for use with entropy coding. It also provides a simple quantization technique for use in noisy channel conditions where a variable length code would be inappropriate but performance greater than that provided by Lloyd-Max quantizers is desired. >

Robust quantization of memoryless sources using dispersive FIR filters

An approach to quantizing discrete-time memoryless sources is presented. An important feature is that its performance is largely insensitive to errors in modeling the input PDF. The method involves changing the amplitude distribution of the source to be approximately Gaussian by all-pass filtering, then applying a Lloyd-Max quantizer designed for a Gaussian source. After quantization, the samples are passed through another all-pass filter, which is an approximate inverse of the first filter. The mean-square error (MSE) for the overall process is roughly equal to the quantization MSE for the intermediate Gaussian signal, independent of the source statistics. For some sources, this is actually an improvement over direct, correct-model Lloyd-Max quantization. The cost of this technique is some delay due to filtering. >

Scalar quantization error analysis for image subband coding using QMFs

The authors analyze the coding errors due to quantization by explicitly incorporating a mathematical model for a Lloyd-Max quantizer into a quadrature mirror filter (QMF) splitting and reconstruction scheme (P. H. Westerink et al., 1988). This approach explicitly incorporates quantization errors into a QMF system by means of a quantizer model. This makes it possible to discriminate between different types of coding errors, such as the aliasing error. Other errors that can be distinguished are a QMF design error, a signal error, and a random error, which is uncorrelated with the original image. Both a mean-squared error calculation and a subjective judgment of the coding errors show that the aliasing errors can be neglected for filter lengths of 12 taps or more. The signal error determines the sharpness of the reconstructed image, while the random error is most visible in the flat areas.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Adaptive block transform coding of speech based on LPC vector quantization

The authors describe several adaptive block transform speech coding systems based on vector quantization of linear predictive coding (LPC) parameters. Specifically, the authors vector quantize the LPC parameters (LPCVQ) associated with each speech block and transmit the index of the code vector as overhead information. This code vector will determine the short-term spectrum of the block and, in turn, can be used for optimal bit allocation among the transform coefficients. In order to get a better estimate of the speech spectrum, the authors also consider the possibility of incorporating pitch information in the coder. In addition, entropy-coded zero-memory quantization of the transform coefficients is considered as an alternative to Lloyd-Max quantization. An adaptive BTC scheme based on LPCVQ and using entropy-coded quantizers is developed. Extensive simulations are used to evaluate the performance of this scheme.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

The root lattices as low bit rate vector quantizers

Vector quantizers based on the A and D lattices are explored, using the mean square error (MSE) fidelity criterion for independent identically distributed sources with normal, Laplacian, and gamma probability density functions. Code books are studied for uniform coding rates of up to 2 b/sample and two and four dimensions. The expected MSE performance of the quantizers is better than that of optimal (nonuniform) Lloyd-Max quantizers. While the design algorithm presented offers only modest improvements in MSE performance for a normally distributed source, improvement gains of up to several decibels can be achieved when coding Laplacian and gamma sources.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A comparison of the Hartley, Cas-Cas, Fourier, and discrete cosine transforms for image coding

The coding method used for the comparison utilizes a marginal bit-allocation scheme and Lloyd-Max quantizer. Gamma, Laplacian, and Gaussian models are compared for the distribution of the transform coefficients. The discrete cosine transform outperforms the other three transforms based on the mean-square quantization error. >

Optimal quantizer design for noisy channels: An approach to combined source - channel coding

We present an analysis of the zero-memory quantization of memoryless sources when the quantizer output is to be encoded and transmitted across a noisy channel. Necessary conditions for the joint optimization of the quantizer and the encoder/decoder pair are presented, and an iterative algorithm for obtaining a locally optimum system is developed. The performance of this locally optimal system, obtained for the class of generalized Gaussian distributions and the binary symmetric channel, is compared against the optimum performance theoretically attainable (using rate-distortion theoretic arguments), as well as against the performance of Lloyd-Max quantizers encoded using the natural binary code and the folded binary code. It is shown that this optimal design could result in substantial performance improvements. The performance improvements are more noticeable at high bit rates and for broad-tailed densities.