Gauss-Markov Source Research Articles

Rate distortion efficiency of subband coding with crossband prediction

Traditional subband coding, where each subband is encoded independently, has been shown by Fischer (1992) to be suboptimum for encoding Gaussian sources in the rate-distortion sense. We show here that if we use prediction across subbands when encoding Gaussian sources, the resulting coder is asymptotically rate-distortion-optimum at high rate. We also show for Gauss-Markov sources that although the crossband prediction filter in this case is of infinite impulse response, the filter can be implemented very efficiently to high accuracy due to the fast decay of the filter coefficients.

Extension of two-stage vector quantization-lattice vector quantization

This paper is the extension of two-stage vector quantization-(spherical) lattice vector quantization (VQ-(S)LVQ) recently introduced by Pan and Fischer (see IEEE Trans. Inform. Theory, vol.41, p.155, 1995). First, according to high resolution quantization theory, generalized vector quantization-lattice vector quantization (G-VQ-LVQ) is formulated in order to release the constraint of the spherical boundary for the second-stage lattice vector quantization (LVQ), which would provide possibilities of improving this kind of two-stage unstructured/structured quantizer by using more efficient LVQ. Second, among G-VQ-LVQ, vector quantization-pyramidal lattice vector quantization (VQ-PLVQ) is developed which is slightly superior or comparable to VQ-(S)LVQ in performance but has a much lower complexity. Simulation results show that for memoryless sources, VQ-PLVQ achieves a rate-distortion performance that is among the best of the fixed-rate quantization that we found in the literature. Therefore, VQ-PLVQ is an attractive alternative to VQ-(S)LVQ in practice. Third, transform VQ-PLVQ (TVQ-PLVQ) is proposed for sources with memory. For encoding 16-D vectors of the Gauss-Markov source, T-VQ-PLVQ has an advantage of close to 1.0 dB over VQ-PLVQ and is about 0.5 dB better than VQ-(S)LVQ.

Graph covering index assignment in vector quantisationfor noisy channels

An index assignment algorithm is presented for vector quantisation coding on noisy channels. The graph covering heuristic requires an enumeration of hypercube connections, for which a constructive procedure is provided. Experimental results for Gauss-Markov sources show that the proposed method provides more reduction in channel distortion than existing techniques.

Necessary conditions for the optimality of variable-rate residual vector quantizers

Necessary conditions for the optimality of variable-rate residual vector quantizers are derived, and an iterative descent algorithm based on a Lagrangian formulation is introduced for designing residual vector quantizers having minimum average distortion subject to an entropy constraint. Simulation results for entropy-constrained residual vector quantizers are presented for memoryless Gaussian, Laplacian, and uniform sources. A Gauss-Markov source is also considered. The rate-distortion performance is shown to be competitive with that of entropy-constrained vector quantization and entropy-constrained trellis-coded quantization.

DPCM system design for diversity systems with applications to packetized speech

Speech quality in packetized speech systems can degrade substantially when packets are lost. We consider the problem of DPCM system design for packetized speech systems. The problem is formulated as a multiple description problem and the problem of optimal selection of the encoder and decoder filters is addressed. We show that significant improvements in performance are obtained as compared to an earlier system proposed by Jayant and Christensen (1981). Further, we show that for a first-order Gauss-Markov source significant performance improvements can be obtained by using a second-order predictor instead of a first-order predictor.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Combined source-channel vector quantization using deterministic annealing

The authors present a new approach to combined source-channel vector quantization. The method, derived within information theory and probability theory, utilizes deterministic annealing to avoid some local minima that trap conventional descent algorithms such as the generalized Lloyd algorithm. The resulting vector quantizers satisfy the necessary conditions for local optimality for the noisy channel case. They tested the method against several versions of the noisy channel generalized Lloyd algorithm on stationary, first order Gauss-Markov sources with a binary symmetric channel. The method outperformed other methods under all test conditions, with the gains over noisy channel GLA growing with the codebook size. The quantizers designed using deterministic annealing are also shown to behave robustly under channel mismatch conditions. As a comparison with a separate source-channel system, over a large range of test channel conditions, the method outperformed a bandwidth-equivalent system incorporating a Hamming code. Also, for severe channel conditions, the method produces solutions with explicit error control coding.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A new tandem source-channel trellis coding scheme

The author presents a new tandem source-channel coding scheme consisting of a trellis source coder and a trellis-coded modulation (TCM) channel coder. The motivation for the use of TCM, instead of conventional channel coding schemes such as convolutional codes, arises from the fact that by using TCM, improvement in the signal-to-quantization-noise ratio (SQNR) is achieved without bandwidth expansion. Criteria for the choice of TCM codes are discussed, and TCM schemes suitable for this application are presented. Simulation results for both a Gauss-Markov source and speech samples indicate that the present scheme results in considerable improvement not only over uncoded transmission but also in comparison with the joint source-channel trellis coding.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Trellis-coded quantization designed for noisy channels

Trellis-coded quantization (TCQ) of memoryless sources is developed for transmission over a binary symmetric channel. The optimized TCQ coder can achieve essentially the same performance as Ayanoglu and Gray's (1987) unconstrained trellis coding optimized for the binary symmetric channel, but with a much lower implementation complexity for transmission rates above 1 b/sample. In most cases, the optimized TCQ coder also provides larger signal-to-noise ratio than Farvardin and Vaishampayan's (1991) channel-optimized vector quantization. Algorithms are developed for the joint design of trellis-coded quantization/modulation (TCQ/TCM). The jointly designed TCQ/TCM system outperforms the straightforward cascade of separately designed TCQ and TCM systems. The improvement is most significant at low channel signal-to-noise ratio. For a first-order Gauss-Markov source, the predictive TCQ/TCM performance can exceed that of optimum pulse amplitude modulation.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Index allocation in vector quantisation for noisy channels

An index allocation algorithm for vector quantisation coding through noisy channels is presented. By applying this index allocation scheme, the mean square error of the communication system due to channel noise can be reduced remarkably without introducing any redundancy. Furthermore, the scheme is rather simple in computation. The experimental results for a Gaussian memoryless source and a Gauss-Markov source are provided.

A structured fixed-rate vector quantizer derived from a variable-length scalar quantizer. II. Vector sources

For Pt.I see ibid., vol.39, no.3, p.851-67 (1993). The fixed-rate scalar-vector quantizer (SVQ) for quantizing stationary memoryless sources is extended to a specific type of vector source in which each component is a stationary memoryless scalar subsource independent of the other components. Algorithms for the design and implementation of the original SVQ are modified to apply to this case. The resulting SVQ, referred to as the extended SVQ (ESVQ), is then used to quantize stationary sources with memory (with known autocorrelation function). Numerical results are presented for the quantization of first-order Gauss-Markov sources using this scheme. It is shown that the ESVQ-based scheme performs very close to entropy-coded transform quantization while maintaining a fixed-rate output and outperforms the fixed-rate scheme that uses scalar Lloyd-Max quantization of the transform coefficients. It is also shown that this scheme performs better than implementable vector quantizers, especially at high rates. >

A unified approach to tree-structured and multistage vector quantization for noisy channels

The large encoding complexity and sensitivity to channel errors of vector quantization (VQ) are discussed. The performance of two low-complexity VQs-the tree-structured VQ (TSVQ) and the multistage VQ (MSVQ)-when used over noisy channels are analyzed. An algorithm is developed for the design of channel-matched TSVQ (CM-TSVQ) and channel-matched MSVQ (CM-MSVQ) under the squared-error criterion. Extensive numerical results are given for the correlation coefficient 0.9. Comparisons with the ordinary TSVQ and MSVQ designed for the noiseless channel show substantial improvements when the channel is very noisy. The CM-MSVQ, which can be regarded as a block-structured combined source-channel coding scheme, is compared with a block-structured tandem source-channel coding scheme (with the same block length as the CM-MSVQ). For the Gauss-Markov source, the CM-MSVQ outperforms the tandem scheme in all cases that the authors have considered. It is demonstrated that the CM-MSVQ is fairly robust to channel mismatch. >

Codebook design for trellis quantization using simulated annealing

Two methods for applying simulated annealing to the trellis quantization codebook design problem are proposed. The results presented for both a Gauss-Markov source and speech samples indicate that the resulting design with simulated annealing is independent of the initial codebook. Moreover, the results are consistent with the generalized Lloyd algorithm with an initial codebook generated using the extension method. This suggests that the extension method is an effective initial codebook selection method. >

Entropy constrained predictive vector quantization of speech

This paper describes a form of predictive vector quantizer that makes use of the entropy constrained quantizer, i.e., an entropy constrained predictive vector quantizer (ECPVQ). An algorithm for designing the ECPVQ is presented, and an approximate theoretical analysis for the performance of the proposed ECPVQ is also provided. It is found that the performance is closely related with the correlation of the input source and the vector dimension. Simulation results on the Gauss-Markov source and real sampled speech data are given and comparisons are made between the new technique presented here and other similar coding techniques, such as entropy constrained VQ and predictive VQ. Dieser Beitrag beschreibt ein Schema eines prädiktiven Vektorquantisierers, der über eine Nebenbedingung in der Entropie beschränkt wird, d.h. einen entropie-beschränkten prädiktiven Vektorquantisierer (entropy-constrained predictive vector quantizer, ECPVQ). Ein Algorithmus für den Entwurf des ECPVQ wird vorgestellt, und eine angenäherte theoretische Analyse wird für die Leistungsfähigkeit des vorgeschlagenen ECPVQ ebenfalls erstellt. Es zeigt sich, daß die Leistungsfähigkeit eng mit der Korrelation der Eingangsquelle und der Vektordimension zusammenhängt. Simulationsergebnisse sind für die Gauß-Markoff-Quelle und echte Sprachdaten aufgenommen, und Vergleiche erfolgen zwischen dem neuen, hier vorgestellten Verfahren und ähnlichen Codierverfahren wie entropie-beschränkter VQ und prädiktiver VQ. Cette article décrit une forme de quantificateur vectoriel prédictif utilisant le quantificateur à contrainte d'entropie, c'est à dire un quantificateur vectoriel prédictif à contrainte d'entropie (ECPVQ). Nous présentons un algorithme de conception de ECPVQ et fournissons une analyse théorique approximative des performances du ECPVO proposé. Nous avons montré que les performances sont étroitement liées à la corrélation de la source d'entrée et à la dimension vectorielle. Nous donnons des résultats de simulation sur une source de Gauss-Markov et des données de signal de parole réelles et nous faisons des comparaisons entre la technique nouvelle présentée ici et d'autres techniques de codage similaires telles que les quantifications vectorielles à contrainte d'entropie et prédictive.

A fast search method based on the statistical properties of signal

The concepts of ordered code-book and the priority of code-vector are proposed is this paper. The statistical properties of a signal are investigated through its coded sequence. Experimental results are presented which provide some insight into the statistical properties of vector quantized sequences. Based on the given concepts and experimental results, a fast search method for the vector quantization of correlated information sources, such as Gauss-Markov sources, is proposed and has shown its efficiency in simulation results.

Source coding and vector quantization with codebook-excited neural networks

The quantization property of layered neural networks is studied in this paper. We first review the layered neural network-based coders or quantizers developed in recent years and show that their poor performances is due to their independent training schemes for each component in the coder. Then an alternative model named a codebook-excited neural network is proposed, where an encoded vector is approximated by the output of the network driven by one vector selected from an excitation codebook. The network and the excitation codebook are jointly trained with the error back-propagation algorithm. Simulations with a Gauss-Markov source demonstrate that the quantization performance of the codebook-excited feedforward neural network is not worse than that of the connectionist vector quantizer formed by a set of single-layer neural units which satisfied the optimal quantization conditions, and that the performance of the codebook-excited recurrent neural network is very close to the asymptotic performance bound of block quantizers. The codebook-excited neural network is applicable with any distortion measure. For a zero-mean, unit variance, memory-less Gaussian source and a squared-error measure, a 1 bit/sample two-dimensional quantizer with a codebook-excited feedforward neural network is found always to escape from the local minima and converge to the best one of the three local minima which are known to exist in the vector quantizer designed using the LBG algorithm. Moreover, due to its conformal mapping characteristic, the codebook-excited neural network can be applied to designing the vector quantizer with any required structural form on its codevectors.

Entropy-constrained trellis-coded quantization

Trellis-coded quantization is generalized to allow noiseless coding of the trellis branch reproduction symbols. An entropy-constrained trellis-coded quantization (ECTCQ) design algorithm is presented, based on the generalized Lloyd algorithm for trellis code design and the entropy-constrained vector quantization design algorithm. ECTCQ systems are designed for memoryless Gaussian and Laplacian sources, and for first- and second-order Gauss-Markov sources. With an 8-state trellis, the mean-squared error ECTCQ encoding performance for memoryless sources with smooth densities is within about 0.5 dB of the rate-distortion function. Small additional improvements are achieved with larger trellises. Similar, though not quite as good, performance is achieved for the Gauss-Markov sources by using the ECTCQ in a predictive coding structure. >

Trellis quantization with MAP detection for noisy channels

Maximum a posteriori (MAP) detection is applied to trellis quantizers operating over additive white Gaussian noise (AWGN) channels. The use of the MAP method instead of maximum likelihood is motivated by the fact that the source coder output probabilities, conditioned on the previous outputs (i.e. the state), are not equal. Simulation results indicate that by using MAP detection instead of maximum likelihood, gains as high as 0.57 dB and 2.2 dB can be achieved in terms of signal-to-quantization noise ratio (SQNR) for Gauss-Markov source and speech samples, respectively.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On the design of connectionist vector quantizers

The Kohonen self-organizing feature map has been widely used for the design of connectionist vector quantizers (VQ). One of the features of the Kohonen algorithm is that the weight update gain sequence η ( m) is a decreasing function of the number of iterations, and if incorrectly chosen can lead to very long training times. Here we derive the time-optimal gain sequence and demonstrate its efficacy for a number of cases. The performance is demonstrated using a Gauss-Markov source and compared with a VQ designed using the LBG algorithm. It is demonstrated that the new method is time optimal and that its performance tends to that of the VQ with the LBG algorithm. The special case of a connectionist VQ for linear predictive or auto-regressive data is analysed. The optimal design is derived for the Itakura distance measures. It is again compared with the LBG design. Two connectionist models for finite-state vector quantizers (FSVQ) are proposed. These models learn not only the distribution but also the conditional distribution of a source. The connectionist FSVQs exploit the redundancy between the vectors (frames) of a highly correlated source, and further improve the rate-distortion performance. Their architectures and learning algorithms are presented. To evaluate the performance of the proposed quantizers, comparisons between quantized and unquantized speech spectra are presented. The connectionist VQ and FSVQ for linear predictive data are finally applied to a multipulse linear predictive speech coder using data from the TIMIT database. Comparisons are made of waveforms and rate-distortion functions.

Performance of entropy-constrained block transform quantizers

An analysis of the rate-distortion performance of an entropy-constrained block transform quantization scheme operating on first-order stationary Gauss-Markov sources is presented. Uniform threshold quantization is employed to quantize the transform coefficients. An algorithm for optimum stepsize (or, equivalently, entropy) assignment among the quantizers is developed and a simple asymptotic formula indicating the high-rate performance of the block transform quantization scheme is presented. Specific results determining the rate-distortion performance of the entropy-constrained block transform quantization scheme operating upon first-order Gauss-Markov sources are presented, and comparisons are made with the Huang and Schultheiss (1963) block transform quantization, vector quantization, and predictive quantization.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

On the performance and complexity of channel-optimized vector quantizers

The performance and complexity of channel-optimized vector quantizers are studied for the Gauss-Markov source. Observations on the geometric structure of these quantizers are made, which have an important implication on the encoding complexity. For the squared-error distortion measure, it is shown that an operation equivalent to a Euclidean distance measurement with respect to an appropriately defined set of points (used to identify the encoding regions) can be used to perform the encoding. This implies that the encoding complexity is proportional to the number of encoding regions. It is then demonstrated that for very noisy channels and a heavily correlated source, when the codebook size is large, the number of encoding regions is considerably smaller than the codebook size-implying a reduction in encoding complexity.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>